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Executive Summary 


The Expository Reading and Writing Course (ERWC) is a grade-12 English course, initially 
developed in 2003/04, to improve the academic literacy of high school seniors, thereby 
reducing the need for students to enroll in remedial English courses upon entering 
college. Developed by California State University (CSU) faculty, high school teachers, and 
high school administrators, the course is a component of a key CSU initiative: the Early 
Assessment Program (EAP). In the EAP, the results of state testing at the end of grade 11 
are used to determine if students are demonstrating readiness for college-level work in 
English and mathematics. The intent of this early signal is to motivate students to use 
grade 12 to become better prepared for college. Intended for such preparation, the ERWC 
emphasizes the in-depth study of expository, analytical, and argumentative reading and 
writing. 

The ERWC consists of a curriculum, professional learning, and curriculum materials. 
These components are designed to lead to teacher instruction and classroom practices 
that promote extended discussion of text meaning and interpretation for the purpose of 
developing students’ critical-thinking skills, that encourage interactive group discussions 
to develop students’ oral language skills, and that enable students to write over extended 
time frames for a range of tasks, purposes, and audiences. The yearlong course includes 
12 modules, or units, of which teachers are expected to teach 8 to 10. 

In 2011, the Fresno County Office of Education (FCOE), in partnership with the CSU and 
WestEd, received an Investing in Innovation (i3) development grant from the 
U.S. Department of Education to update and refine the ERWC and to augment its 
program of professional learning. WestEd was funded to conduct a rigorous 
independent evaluation of the course’s impact on student achievement and to assess the 
fidelity of course implementation at study sites. The outcome measure used in 
evaluating the course’s impact was the English Placement Test (EPT), a standardized test 
given to students entering the CSU system who have not demonstrated English 
proficiency by other measures, in order to determine their eligibility for placement into a 
credit-bearing college English course. 

The primary confirmatory research question for the impact evaluation was “Does the 
Expository Reading and Writing Course have a positive impact on the reading and writing 
skills of grade-12 students as measured through the English Placement Test?” The 
primary research question for the assessment of implementation fidelity was “What 
proportions of study teachers received all curriculum materials, attended a sufficient 
number of the professional-learning opportunities, and taught with fidelity a sufficient 
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number of the curriculum’s modules?” This executive summary presents the findings of 
the impact evaluation and the assessment of implementation fidelity. 

Impact Evaluation Methodology and Study Sample 

To estimate the impact of the ERWC on student achievement, a matching analysis was 
performed, in which grade-r2 students enrolled in the ERWC were matched to similar 
grade-12 students who were not enrolled in the ERWC; the comparison students, referred 
to as “non-ERWC students,” were generally enrolled in either an English 4 course or an 
Advanced Placement (AP) English literature course. ERWC students were matched to 
non-ERWC students based on the following student characteristics: grade-11 English 
language arts (ELA) California Standards Test (CST) scale score, grade-11 AP English 
course enrollment, average grade-11 English grade earned, gender, and ethnicity. 

Key aspects of the matching process were: the Mahalanobis distance metric was used as 
the measurement of how similar ERWC students and non-ERWC students were; each 
ERWC student was matched to the four most similar non-ERWC students (“one-to-many” 
matching); and matching was conducted with replacement (i.e., non-ERWC students were 
allowed to be a match for more than one ERWC student). 

Non-ERWC students were weighted in the final analytic sample, due to the fact that a 
non-ERWC student may be included in the sample a number of times as a result of being 
matched to multiple ERWC students. The final analytic sample consisted of 3,309 ERWC 
students who were matched with 3,309 weighted non-ERWC students. 

After the matching was conducted, all matched students were included in a regression 
model that included as covariates the same variables that were used in the matching 
process. As previously noted, the outcome measure used in the regression model was the 
EPT; thus, all students who participated in the evaluation were asked to take the EPT. 

Estimated Impact on Student Achievement 

Compared to non-ERWC students, students who enrolled in the ERWC scored higher on 
the EPT, and the difference was statistically significant at the r percent level. This means 
that the ERWC was found to have a positive impact on student achievement, and that this 
impact was unlikely to have happened by chance. The estimated standardized effect of 
enrollment in the ERWC was .13 standard deviations. In addition, the positive, statistically 
significant results were robust to a number of different sensitivity analyses. These 
sensitivity analyses included varying the methods used to match ERWC and non-ERWC 
students and varying the types of students included in the analytic sample. 
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Fidelity of Implementation of the ERWC 

For a teacher to be considered as having implemented the ERWC with fidelity, he or she 
must have taught, with fidelity, a minimum of 8 of the course’s 12 curriculum modules. Of 
the 56 ERWC teachers who participated in the study, 10 (17.9 percent) taught at least 
8 modules with fidelity. This low percentage is due to the stringent requirement that, for a 
teacher to be considered as having taught any module with fidelity, he or she needed to 
teach at least one activity in each of the module’s six strands (i.e., Prereading; Reading; 
Postreading; Discovering What You Think; Entering the Conversation; and Revising and 
Editing) . Study teachers were not made aware that this criterion would be used to 
determine if a teacher had taught the course with fidelity. If, instead, the criterion had 
been defined as teaching at least five activities in a given module, irrespective of the 
strand — an approach that would provide a better indication of the number of teachers 
who completed 8 modules over the course of the year — 35 teachers (62.5 percent) would 
have been defined as having taught the course with fidelity. But regardless of the criterion 
used to define whether or not a teacher taught a particular module with fidelity, many 
teachers provided reasons for not having attempted to teach 8 modules during the year. 
Among the reasons were decisions to supplement modules with additional texts that were 
deemed helpful to the students; district-mandated assessments that took up class time 
throughout the year; and some students not having the necessary foundational skills that 
would allow them to keep up with the ERWC’s rigorous demands if the teacher moved at a 
pace necessary to complete at least 8 modules during the year. 

For the teachers involved in the ^-funded study, the professional-learning component of 
the ERWC consisted of a two-day summer professional-learning session, professional 
learning community meetings, and coaching sessions. With respect to the professional- 
learning component, 46 of 56 teachers (82.1 percent) were deemed to have participated in 
a sufficient number of professional-learning opportunities. With respect to receiving the 
ERWC curriculum materials, each of the 56 study teachers received all of the materials. 

Exploratory Findings from Qualitative Survey Data 

This report also presents exploratory findings from open-ended responses from the data- 
collection instruments completed by study participants. These findings are meant to 
provide context about what happened during the study period and to help shape the 
direction of future research on the ERWC. Among the findings from this exploratory 
research are factors that were identified as hindering implementation of the curriculum. 
For instance, teachers commonly reported struggles in being able to sustain the pace that 
would be needed for them to complete at least 8 modules during the course of a school 
year. In addition, many students struggled with the rigors of the ERWC, particularly at the 
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beginning of the school year. Other teachers noted technology impediments that 
prevented them from completing certain activities. 

Discussion 

While the implementation-fidelity analysis found that many teachers did not complete at 
least one activity in each strand of the modules they taught, the impact evaluation found 
that the ERWC had a positive effect on student achievement. Therefore, future evaluations 
could assess whether impacts of the course would be even greater if teachers were 
explicitly directed to teach at least one activity in each strand of every module taught. 
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Chapter 1 . Introduction 


The Investing in Innovation Development Grant for the 
Expository Reading and Writing Course 

In 2011, the Fresno County Office of Education (FCOE), in partnership with the California 
State University (CSU) and WestEd, received an Investing in Innovation ( 13 ) development 
grant from the U.S. Department of Education to update and refine the Expository Reading 
and Writing Course (ERWC). The ERWC is a grade-12 English course, initially developed 
in 2003/04 by a task force of CSU faculty and high school educators from around 
California, to improve the academic literacy of high school seniors, thereby reducing the 
need for students to enroll in remedial English courses upon entering college. 1 Using 
mostly, though not exclusively, nonfiction texts, the ERWC emphasizes the in-depth study 
of expository, analytical, and argumentative reading and writing. 

The i3 development grant funded the FCOE and the CSU to (1) update and refine the 
curriculum materials and (2) increase the scope and effectiveness of the ERWC 
professional learning. WestEd was funded to conduct a rigorous independent evaluation 
of the ERWC’s impact on student achievement and to assess the fidelity of the ERWC 
implementation at study sites. The outcome measure used in evaluating the course’s 
impact was the English Placement Test (EPT), a standardized test given to students 
entering the CSU system who have not demonstrated English proficiency by other 
measures, in order to determine their eligibility for placement into a credit-bearing college 
English course. The primary confirmatory research question for the impact evaluation was 
“Does the Expository Reading and Writing Course have a positive impact on the reading 
and writing skills of grade-12 students as measured through the English Placement Test?” 
The primary research question for the assessment of implementation fidelity was “What 
proportions of study teachers received all curriculum materials, attended a sufficient 
number of the professional-learning opportunities, and taught with fidelity a sufficient 
number of the curriculum’s modules?” The ERWC was implemented in study sites for this 
evaluation in the 2013/14 school year; this report documents the findings of the impact 
evaluation and the assessment of implementation fidelity. The report also provides 
exploratory findings based on open-ended responses from the ERWC teachers and 


1 High rates of college remediation in California, and throughout the nation, are well documented (see, 
for instance, Sparks & Malkus, 2013; Strong American Schools, 2008; Parsad & Lewis, 2003). In the case of 
California, in fall 2010 — the year prior to the 13 development grant being awarded for the ERWC — the 
proportion of regularly admitted full-time freshmen needing remediation in English in the CSU system 
was 49.3 percent; as of fall 2013, that rate had declined to 32.1 percent (California State University, 2015). 
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coaches; these exploratory findings help to provide context about the implementation of 
the course. 

History, Theory, and Prior Research on the Expository Reading 
and Writing Course 

ERWC History 

The ERWC was initially developed in 2003/04 in response to a growing concern that 
graduating high school seniors were not equipped with the academic skills needed to 
succeed in college. In particular, course developers — a task force of CSU faculty and 
California high school educators — were responding to findings that high school seniors 
needed to improve in the areas of analytical and expository reading and writing. For 
instance, findings from a report by the Intersegmental Committee of the Academic 
Senates (2002) showed that 83 percent of surveyed college faculty said that students’ lack 
of analytical reading skills contributed to their lack of success in a course. The same report 
also found that only one third of entering college students were sufficiently prepared for 
the two most frequently assigned writing tasks: analyzing information or arguments and 
synthesizing information from several sources. 

The ERWC was first piloted in the 2004/05 school year with approximately 660 California 
high school English teachers (California State University, 2005). The piloting stage 
continued through 2007, after which the course was revised in response to constructive 
feedback. The revised course was then published in 2008 for use by schools throughout 
the state. A second version of the course, updated and refined through use of i3 
development grant funds, was released for the 2013/14 school year. The course is now 
widely offered to grade-12 students throughout California. To date, more than 
750 California high schools have adopted the ERWC as a full-year grade-12 English course. 
Its popularity was such that, in the 2011/12 school year, the ERWC was assigned its own 
California Basic Educational Data System (CBEDS) identification number of 2118. 2 

ERWC Theory 

The ERWC’s overarching goal is to equip students with strong critical reading and writing 
skills. The course is based on seven key principles: 

1. The integration of interactive reading and writing processes; 


2 CBEDS assignment codes are standardized four-digit codes used by all public schools in California to 
reflect the curriculum covered in each course. 
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2. A rhetorical approach that fosters critical thinking and engagement through a 
relentless focus on the text; 

3. Materials and themes that engage student interest; 

4. Classroom activities designed to model and foster successful practices of fluent 
readers and writers; 

5. Research-based methodologies with a consistent relationship between theory 
and practice; 

6. Built-in flexibility to allow teachers to respond to varied students’ needs and 
instructional contexts; and 

7. Alignment with the California Common Core State Standards for English 
Language Arts and Literacy. 


Prominent in the course curriculum is a focus on reading and writing rhetorically. ERWC 
students receive extensive practice analyzing and applying the three classical categories of 
rhetorical appeal: ethos (i.e., appeal to credibility), logos (i.e., appeal to logic), and pathos 
(i.e., appeal to emotion). 

Curriculum modules, or units, are organized by three major domains: Reading 
Rhetorically; Connecting Reading to Writing; and Writing Rhetorically. With respect to 
Reading Rhetorically, students are taught to focus not just on what the text says but also 
on the purposes it serves, the author’s intentions, and the text’s effects on the audience. As 
each new module is introduced, Reading Rhetorically begins with the Prereading strand, 
during which readers prepare to read the new text. In this strand, students survey the text 
and consider its purpose, context, author, form, and language. This process helps students 
establish a purpose and plan for reading, anticipate what the text will discuss, and 
establish a framework for understanding the text. They use the knowledge developed 
during this prereading phase to help them understand the text as they proceed to the 
Reading strand of the module. As they read, the new knowledge helps them confirm, 
refine, or refute their prior predictions. The Reading strand is followed by the Postreading 
strand, which can involve restating the central ideas of the text and responding to them 
from a personal perspective; it can also include questioning the text and its rhetorical 
strategies, evaluating its arguments and evidence, and considering how it fits into the 
larger conversation about a topic. Each such strand, across all domains, includes elements, 
with related activities, that need to be taught and learned if students are to be able to 
master the given strand and, ultimately, its larger domain. The analytic skills developed in 
this Reading Rhetorically domain build students’ capacities to discern whether the texts 
they read are credible. 

In the Connecting Reading to Writing domain, students begin to consider the writing task 
and start to gather evidence to support claims that will be made in the writing phase of 
the module. In the Writing Rhetorically domain, students also consider the importance of 
audience, purpose, situation, and genre to what they are writing. The writing in the 
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curriculum is “reading-based” in that it synthesizes, for the writer’s own purposes, the 
viewpoints and information of various sources. Students are taught how to properly use 
information from the texts they have read to support their written arguments. The ERWC 
also emphasizes the importance of writing as a form of communication; while writing can 
allow students to discover what they are thinking and get them to work through their 
personal concerns and ideas, the course also has students write to express their ideas to 
others. (For a full list of the strands in the Connecting Reading to Writing and Writing 
Rhetorically domains, see table 2.2.) 

Throughout the ERWC, students practice multi-tiered academic writing in numerous 
ways: 

• They write 750- to 1,500-word analytical essays based on prompts that require 
them to establish and develop a thesis or argument and to provide evidence in 
support of that thesis by synthesizing and interpreting the ideas presented in 
texts. 

• They complete timed in-class writing assignments based on prompts related to 
the authors’ assertions, themes, purposes, or rhetorical techniques. 

• They complete a summative writing assignment in each module. 

• They prepare a writing portfolio for a final reflection on learning. 

The ERWC also integrates oral language development through such activities as class 
discussion, presentations, dialogues, role play, and pair conversations. Students are taught 
to listen and speak with a heightened awareness of audience, occasion, purpose, and 
persuasive strategies, similar to how they are taught to read and write rhetorically. They 
are also taught to respond respectfully to divergent views, to listen both supportively and 
critically, and, when speaking, to consider the needs and interests of their audience. 

Prior Research on the ERWC 

In the past decade, a number of small-scale impact studies of the ERWC have been 
conducted. These studies, briefly described in this section, provided encouraging findings 
about the course. 

In 2005, a study in California examined the relationship between students’ exposure to the 
ERWC and their reading and composing proficiency at the end of grade 12 (California 
State University, 2005). The study included a sample of 10 teachers who served as the 
treatment group, with the requirement that each teacher needed to have taught at least 
two of the ERWC modules during the school year. The control teachers consisted of a 
group of English teachers, chosen by the treatment teachers, who all taught a traditional 
grade-12 college-preparatory English course. At the end of the school year, all teachers in 
the study administered to their grade-12 students the Reading and Composing Skills Test 
(RCST), a 40-item nonsecure assessment that consists of retired items from the EPT and 
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that can be administered during a single class period. Based on the 225 students who took 
the RCST (r3o treatment students and 95 control students), the results of a t-test indicated 
a positive and statistically significant relationship between exposure to at least two of the 
ERWC modules and performance on the RCST. Qualitative results from survey data found 
that teachers viewed the curriculum materials as academically rigorous and engaging for 
their students. 

A 2007 study examined the relationship between enrollment in the ERWC and student 
achievement (Elafner & Joseph, 2007). In this study, 12 ERWC schools with a total of 
62 teachers who had participated in ERWC professional learning were identified and 
matched to comparison schools, based on school size, proportion of students qualifying 
for free or reduced-price lunch, and proportion of students who were English language 
learners. Based on the comparison between the 12 ERWC schools and 18 matched control 
schools, the authors report that the average increase in the percentage of proficient 
students in English at ERWC schools was three times the average increase at control 
schools between fall 2003 and fall 2006 (11 percent versus 3.6 percent). 3 The study also 
found that participating ERWC teachers reported high levels of satisfaction with the 
curriculum and related professional development. 

In 2oro, Hafner, Joseph, and McCormick published a study on the ERWC. The study 
yielded a variety of survey-based findings indicating broad support for the course. For 
example, participating teachers reported that the course had a positive impact on 
students’ reading and writing skills, motivation, and increased time-on-task, all of which 
are associated with improvements in English proficiency. Quantitative student outcomes 
were collected to measure school-level outcomes on graduation rates in 2008, gains on the 
Academic Performance Index (API) between 2004 and 2008, and gains in the percent of 
proficient students between 2004 and 2008 as measured by the grade-rr English Language 
Arts (ELA) California Standards Test (CST). The authors report that the five study schools 
outperformed the statewide averages with respect to graduation rate (study schools had 
an average rate of 90 percent, versus a statewide rate of 80 percent), API gains (study 
schools had an average gain of 69 points, versus a 3r-point gain statewide), and ELA CST 
gains (study schools experienced a 7-percentage-point gain, versus a 4-percentage-point 
gain statewide). 

Taken together, these three studies provide suggestive evidence that enrollment in the 
ERWC is associated with gains in ELA achievement. However, these earlier studies had 
small sample sizes, and any matching that was conducted between ERWC and non-ERWC 
students was conducted at the school level. The current evaluation was undertaken to 


3 The authors defined “proficient” students as those at the high school who met any of the following 
criteria: were exempt from taking the EPT, scored proficient on the EPT, or demonstrated proficiency 
prior to enrollment in the CSU system. 
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evaluate a large number of schools; to perform matching at the student level, using a wide 
variety of demographic and prior-achievement variables; to ensure baseline equivalence 
between the ERWC and non-ERWC students studied; and to conduct an analysis using a 
rigorous and standardized outcome measure. These study characteristics allow for 
stronger inferential statements to be made about the impact of the ERWC on student 
achievement. 
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Chapter 2. The Expository Reading 
and Writing Course 


This chapter provides a detailed discussion of the Expository Reading and Writing Course 
(ERWC) as implemented in the evaluation. It includes detailed descriptions of the key 
course components: the curriculum, the professional learning, and the curriculum 
materials. Figure 2.1 presents the logic model used in the evaluation. In addition to 
showing the key inputs and outputs for the course, the logic model identifies the course’s 
underlying rationale — namely, that many students enter college without the necessary 
academic skills to succeed. The ERWC seeks to mitigate this problem by equipping grade- 
12 students with strong critical reading and writing skills through a curriculum that 
focuses primarily on expository text. 

The key components, or inputs, of the ERWC are the curriculum, the professional 
learning, and the curriculum materials. These inputs are expected to influence teacher 
instruction and classroom practices that include extensive discussions of text meaning, 
with the intention of developing students’ critical-thinking skills; the use of materials that 
are highly engaging to students; and extensive writing assignments that cover a range of 
tasks, purposes, and audiences. These classroom practices are expected to result in 
improved student outcomes, such as students’ ability to produce clear and coherent 
writing, which can be measured through the English Placement Test (EPT). 

The following sections further describe the three ERWC inputs. 
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Figure 2.1. The Expository Reading and Writing Course Logic Model 
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For Students 
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-> 
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Interactive group discussions that develop oral 
language skills 


Outcomes 
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KEY 


A single asterisk (*) identifies components that 
were assessed for fidelity of implementation. 


Double asterisks (**) identify student outcomes that 
were used for the confirmatory impact evaluation. 


Triple asterisks (***) identify student outcomes that 
were used for the exploratory impact evaluation. 


Source: Authors. 
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ERWC Curriculum 


The ERWC curriculum includes 12 modules, each covering a different topic, with teachers 
expected to teach 8 to 10 of the modules over the course of the school year. Module 
numbers and titles are provided in table 2.1. Each topic was chosen, and its module 
designed, to pique students’ interest. The first module of the course is titled “What’s Next? 
Thinking About Life After High School.” Other module topics include, for example, racial 
profiling, juvenile justice, and bullying. Three of the modules are based on full-length 
books: Into the Wild by Jon Krakauer (module 6), 1984 by George Orwell (module 10), and 
Brave New World by Aldous Huxley (module 11). Teachers are to teach one full-length 
book in each semester; in other words, teachers should teach module 6 in the first 
semester and either module 10 or module 11 in the second semester. For the 8 to 10 
modules that teachers choose to teach, the teachers are encouraged to teach them in 
sequence (e.g., module 2 before module 5) and to consider the balance of text types and 
writing assignments across the modules. 


Table 2.1. Expository Reading and Writing Course Modules 


Semester One 

Semester Two 

Module 1 : What's Next? Thinking About Life 
After High School 

Module 7: Bring a Text You Like to Class: 
Bridging Out-of-School and In-School 
Literacies 

Module 2: The Rhetoric of the Op-Ed Page: 
Ethos, Logos, and Pathos 

Module 8: Juvenile Justice 

Module 3: Racial Profiling 

Module 9: Language, Gender, and Culture 

Module 4: The Value of Life 

Module 10: 1984 

Module 5: Good Food/Bad Food 

Module 1 1 : Brave New World 

Module 6: Into the Wild 

Module 1 2: Bullying: A Research Project 


Source: The Expository Reading and Writing Course curriculum. 


All modules follow a similar structure that is based on the ERWC’s Assignment Template, 
which presents a scaffolded process to help students read, comprehend, and respond to 
nonfiction and literary texts. As shown in table 2.2, each module consists of three 
overarching domains: Reading Rhetorically, Connecting Reading to Writing, and Writing 
Rhetorically. Each domain includes strands. The Reading Rhetorically domain includes 
three strands: Prereading, Reading, and Postreading. The Connecting Reading to Writing 
domain consists of one strand: Discovering What You Think. Finally, the Writing 
Rhetorically domain consists of two strands: Entering the Conversation and Revising and 
Editing. 
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As noted previously, within each strand are elements that need to be taught and learned if 
students are to be able to master the given strand and, ultimately, its larger domain. For 
instance, the Prereading strand consists of the following elements: Getting Ready to Read; 
Exploring Key Concepts; Surveying the Text; Making Predictions and Asking Questions; 
and Understanding Key Vocabulary. In the ERWC curriculum, these elements contain the 
activities that help students develop the skills in a given area. As an example, the 
Surveying the Text element in a particular module may include an activity that asks 
students to answer a number of questions about the texts they are going to read. If a 
teacher engages his or her students in this activity, then the teacher will have taught this 
activity within the Prereading strand of the Reading Rhetorically domain . 4 

Also embedded within the ERWC Assignment Template are recommendations for 
formative assessment processes that promote ongoing evaluation of student progress 
toward learning objectives, as well as rhetorical grammar activities that support students 
in their acquisition of academic English. The rhetorical grammar activities use curriculum 
readings and students’ own writings to build competence in writing conventions and 
language use. 


4 In some modules, a given element may contain multiple activities. In other modules, a given element 
may not include any activities for the teacher to complete. 
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Table 2.2. Overview of the Expository Reading and Writing Course Assignment Template 


Reading 

Rhetorically 

Reading 

Rhetorically 

Reading 

Rhetorically 

Connecting 
Reading to Writing 

Writing 

Rhetorically 

Writing 

Rhetorically 

Prereading 

Reading 

Postreading 

Discovering What 
You Think 

Entering the 
Conversation 

Revising and 
Editing 

• Getting Ready to 
Read 

• Exploring Key 
Concepts 

• Surveying the Text 

• Making Predictions 
and Asking 
Questions 

• Understanding Key 
Vocabulary 

• Reading for 
Understanding 

• Considering the 
Structure of the Text 

• Noticing Language 

• Annotating and 
Questioning the Text 

• Analyzing Stylistic 
Choices 

• Summarizing and 
Responding 

• Thinking Critically 

• Reflecting on Your 
Reading Process 

• Considering the 
Writing Task 

• Taking a Stance 

• Gathering 
Evidence to 
Support Your 
Claims 

• Getting Ready to 
Write 

• Composing a 
Draft 

• Considering 
Structure 

• Using the Words of 
Others (and 
Avoiding 
Plagiarism) 

• Negotiating 
Voices 

• Revising 
Rhetorically 

• Considering 
Stylistic Choices 

• Editing the Draft 

• Responding to 
Feedback 

• Reflecting on Your 
Writing Process 


Source: The Expository Reading and Writing Course curriculum. 
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ERWC Professional Learning 

In addition to the initial 20 hours of professional learning in which all ERWC teachers 
participate to be eligible to teach the course, ERWC-study teachers participated in 
updated professional learning throughout the school year. The updated professional 
learning for the ERWC teachers in the study consisted of three subcomponents: a two-day 
summer professional-learning session, professional learning community (PLC) meetings, 
and coaching sessions. 

During the initial professional learning for all ERWC teachers, teachers learned how to 
integrate effective reading and writing pedagogy into their instruction; how to integrate 
reading and writing with a focus on engaging, debate-worthy themes and texts that 
students would find motivating; how to provide instructional scaffolding to help a range 
of students succeed in reading and writing sophisticated texts and concepts; and how to 
create an environment in which the opinions of teachers and students are both sought 
and respected. 

The first professional-learning subcomponent for teachers in the study, the two-day 
summer session, led by ERWC curriculum developers , 5 familiarized the teachers with the 
newly updated modules and offered pragmatic strategies for using the curriculum 
materials. Topics for the session also included protocols for looking at student work, 
strategies for handling the large amount of student writing to be graded for the class, 
discussion topics for the PLC meetings, and an overview of the coaching sessions. 

The second subcomponent of the professional learning was the PLC meetings, which 
provided an opportunity for teachers to discuss their practice in a reflective, collaborative, 
and growth-promoting way. Within the PLCs, the teachers were expected to share 
successful instructional strategies, identify gaps in student knowledge, collaboratively 
design interventions and strategies for students not demonstrating sufficient learning 
gains, and identify future instructional goals. Teachers were to participate in at least nine 
PLC meetings throughout the school year. 

The third subcomponent of the professional learning was coaching. In this subcomponent, 
ERWC coaches, who were either CSU faculty or curriculum specialists from county offices 
of education, observed teachers teaching the curriculum and then, in coaching sessions, 
provided feedback to help the teachers with instruction. (They also occasionally attended 
PLC meetings to assist as needed.) During the coaching sessions teachers could work with 


5 This two-day professional-learning session for the 13 evaluation is different from the initial 20 hours 
(three to three and a half days) of professional learning that teachers originally attended when they first 
became certified to teach ERWC. The ERWC developers refer to the 13 two-day session for already- 
certified ERWC teachers as “PL Plus.” 
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coaches to identify areas for pedagogical improvement, develop refined plans to improve 
student learning, discuss strategies to further engage students with the texts, and improve 
instruction of student writing. The coaching sessions also allowed teachers to have a 
chance to reflect on their experiences teaching the ERWC curriculum and to ask any 
questions. Teachers were to participate in at least four coaching sessions during the 
school year. 

ERWC Materials 

The ERWC curriculum materials that were updated in 2013 included the following 
components for each module: a teacher version, a student version, and a set of student 
readings (including full-length books for two modules 6 ). For nine of the modules, a 
teacher version and a student version of accompanying rhetorical grammar lessons were 
provided. The curriculum modules were divided into two semesters, each contained in a 
binder. The ERWC: Semester One (2nd edition) binders were distributed to the study 
teachers at the two-day summer professional-learning session. 7 The printing of the ERWC: 
Semester Two (2nd edition) binders was delayed, and the binders were subsequently 
mailed to each high school participating in the study. 


6 There are three full-length books in the curriculum, but teachers were only expected to teach two of 
these books (one per semester). As a result, each teacher received two full-length books for each student 
in his or her class. 


7 For each of the two ERWC teachers in the study who were not able to attend the summer professional- 
learning session, the teacher version of the curriculum was mailed to the high school where the teacher 
taught. 
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Chapter 3. Impacts of the 
Expository Reading and Writing 
Course on Student Achievement 


This chapter presents results of the evaluation of the Expository Reading and Writing 
Course (ERWC)’s impact on academic achievement. The chapter begins with a discussion 
of the evaluation methodology and then describes the data used in the evaluation, 
followed by a discussion of the outcome measure — the English Placement Test (EPT) — 
used in the evaluation. Next, results of the baseline balance tests are presented. The 
overall findings of the impact evaluation are then provided. Finally, the results from a 
number of sensitivity analyses are discussed. 

Impact Evaluation Methodology 

This impact evaluation employed a matching analysis in its analytic design (see, for 
instance, Rosenbaum & Rubin, 1983; Caliendo & Kopeinig, 2008; Steiner & Cook, 2013; 
Huber, Lechner, & Wunsch, 2013; Imbens, 2015). This entailed analytically matching 
ERWC students with similar students who did not enroll in the ERWC. This latter group 
of comparison students is referred to throughout the report as “non-ERWC students.” 
ERWC students were matched to similar non-ERWC students based on the Mahalanobis 
distance metric, which is defined as the distance between two values of the covariate 
vector x and x’\ | \x, x' | | = (x — x')i7^ 1 (x — x') where Q x is the sample covariance matrix of 
the covariates (see, for instance, Imbens, 2015). The matching was conducted in 
R statistical software (R Core Team, 2014), using the Match It command (Ho, Imai, King, & 
Stuart, 2011). 8 9 10 

In the matching analysis, each ERWC student was matched to the four closest non-ERWC 
students (“nearest neighbors”) such that the matching was conducted “one-to-many.” 9, 10 


8 R version 3.1.2 was used to conduct the analysis. 

9 The pre-analysis plan for this evaluation specified a “one-to-many” matching approach, but the exact 
number of matches was not pre-specified. Using more matches for each treatment observation will 
improve statistical precision by increasing the total sample size, but it may also lead to more bias 
because more dissimilar matches will be included. Our final estimation was based on matching each 
ERWC student to four non-ERWC students. The section of this chapter that presents the sensitivity 
analyses shows the results from varying the number of matches for each ERWC student; the results were 
robust to the number of matching non-ERWC students used. 

10 The matching was not done using an iterative process. More specifically: Sometimes, in matching 
analyses, the analyst will perform the preprocessing matching and then check the baseline equivalence. 
If baseline equivalence is not achieved, the analyst will adjust the matching model so that treatment 
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The matching procedure only matched ERWC students to similar non-ERWC students; 
each non-ERWC student was not also matched to a similar ERWC student. Thus, the 
resulting estimate is the average treatment effect for the treated. * 11 Matching was 
conducted “with replacement,” such that each non-ERWC student could be a match to 
multiple ERWC students if that non-ERWC student was similar to multiple ERWC 
students. 12 Students from different school districts were allowed to be matched because 
the primary outcome measure (the EPT) is not impacted by district policies. 

The following variables were used to match ERWC students to similar non-ERWC 
students: grade-11 English Language Arts (ELA) California Standards Test (CST) scale 
score, 13 grade-11 Advanced Placement (AP) English course enrollment (a binary variable 
equal to one for students whose grade-11 English course enrollment was in AP English and 
equal to zero otherwise), 14 average grade-11 English grade earned, 15 gender, and ethnicity. 
The student ethnicity categories used were African American, Asian, Hispanic, and White. 

After each ERWC student was matched to the four closest non-ERWC students, all of the 
matched students were included in an ordinary least squares (OLS) regression model that 
included the same variables as covariates that were used in the matching process. This was 
done to make the evaluation more robust in that the matching and the regression protects 
against misspecification in either model (Imbens & Wooldridge, 2009). Previous studies 
have suggested that matching on a set of baseline data that are strongly predictive of the 
outcome measure and then using regression methods on the matched sample can succeed 


students are matched to different comparison students, and will then check the baseline equivalence 
again; the analyst will run different models until baseline equivalence is achieved. However, in our 
analysis, baseline equivalence was achieved after the first match. 

11 Our interest was not in estimating the average treatment effect for the overall population, which would 
additionally require matching control students with observationally similar treatment students in order 
to impute each control student’s missing treatment outcome (Steiner & Cook, 2013). 

12 Matching with replacement reduces bias by allowing for more similar matches, but it leads to less- 
precise estimates. 

13 The grade-11 ELA CST was a standardized assessment that California students took in the spring of 
their grade-11 year to assess their knowledge of the California content standards in reading and writing. 
The assessment contained 75 multiple-choice questions, and scale scores ranged from a low of 150 to a 
high of 600. With the introduction of the Common Core State Standards, the ELA CST is no longer 
administered as of the 2014/15 school year. 

14 Originally, in the pre-analysis plan, this variable was to indicate enrollment in grade 11 in AP English 
Language, International Baccalaureate (IB) English, or honors English. However, inspection of the data 
from the school districts included in this analysis indicated that grade-n students did not enroll in IB 
English or honors English. 

15 This variable was calculated by first converting letter grades earned in each semester to a numeric scale 
and then averaging the numeric values. The numeric scale was as follows: “A” = 4.0, “A-” = 3.67, 

“B+” = 3.33, “B” = 3.0, “B-” = 2.67, “C+” = 2.33, “C” = 2.0, “C-” = 1.67, “D+” = 1.33, “D” = 1.0, “D-” = 0.67, “F+” 

= 0.33, and “F” = 0.0. 
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in replicating experimental impacts in certain contexts (Cook, Shadish, & Wong, 2008; 
Fortson, Gleason, Kopa, & Verbitsky-Savitz, 2014; Furgeson et ak, 2012; Gill et ak, 2013). 16 
The regression model was of the following form: 

EPT, = a + fiiERWQ) + $ 2 (ELA_CST) + (3 3 (ELA_GPA>) + ^(Ethnicity,) + p 5 (AP I ) + 

\i ( ,( Female,) + £ ; (1) 

where EPT X is student i’s EPT scale score, ERWC is a binary variable indicating enrollment 
in the ERWC, ELA_CST is the grade-11 ELA CST scale score, ELA_GPA is the average 
English grade earned in grade 11, Ethnicity is a vector of dichotomous variables indicating 
student i’s ethnicity (African American, Asian, Flispanic, or White, with African American 
being the omitted variable in the regression), AP is a binary variable identifying whether 
or not a student enrolled in AP English Language in grade 11, and Female is a binary 
variable identifying female students, a is the intercept, are parameters to be 
estimated from the data, and e is the independent and identically distributed error term. 

P, in equation (1) represents the average difference in EPT scale scores between ERWC and 
non-ERWC students after controlling for the covariates included in the model; this 
parameter represents the impact of the ERWC curriculum. 

In this regression equation, each ERWC student received a weight of 1, while each 
matched non-ERWC student received a weight that was proportional to the number of 
times the non-ERWC student was matched. Because each ERWC student was matched to 
four non-ERWC students, each non-ERWC student received a weight of 0.25 for each time 
he or she was used as a match. This weighting scheme was necessary to ensure that the 
four matched non-ERWC students collectively had the same weight as the one ERWC 
student that they were matched to. If, for instance, a non-ERWC student had been 
matched to three different ERWC students, then that non-ERWC student received a 
weight of 0.75 (= 0.25 * 3) in the OLS regression. The total sample sizes of each group 
(ERWC and non-ERWC) were equivalent after the weights were applied. Because 
non-ERWC students could be included multiple times in the regression analysis, 
cluster-robust standard errors (Fluber, 1967) were used, to allow for intragroup correlation 
at the individual level. The use of robust standard errors when conducting a regression 
after matching with replacement is suggested by Hill and Reiter (2006, p. 2234). 17 


16 Similar to these previous studies, after matching was completed we performed an OLS regression (as 
opposed to hierarchical linear modeling). See, for instance, Fortson, Gleason, Kopa, & Verbitsky-Savitz 
(2014), Gill et al. (2or3), and Furgeson et al. (2or2) for examples of OLS regression being performed after 
matching. An OLS regression, as opposed to hierarchical linear modeling, is conducted after matching 
because it is assumed that assignment to the treatment or control status occurs at the student level 
rather than the teacher or school level (which is one reason why matching was conducted at the student 
level as well). OLS regression was also specified in this evaluation’s pre-analysis plan. 

17 Matching without replacement would avoid the situation of having non-ERWC students being 
included multiple times in the regression analysis. In this evaluation, sensitivity analyses were conducted 
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Data 


Schools were originally recruited into the study if they met all of the following criteria at 
the time of recruitment: (1) the school must have had teachers who had taught the ERWC 
in the past, (2) the school must have had teachers who were willing to participate in the 
study and complete the teacher surveys, (3) the school must have been willing to 
administer the EPT during the spring semester, and (4) the school must have been part of 
a district that was willing to provide student-level data. A total of 24 high schools in nine 
school districts across California met these criteria and were selected to participate in the 
study. These high schools exhibited wide variation in terms of school locales, academic 
performance, and geographic locations. With respect to school locale, 9 schools were 
located in a “City, large territory”; 4 schools were located in a “City, mid-size territory"; 

2 schools were located in a “City, small territory”; 6 schools were located in a “Suburb, 
large territory”; and 3 schools were located in a “Rural, fringe census.” 18 With respect to 
academic performance, the schools scored in the following academic deciles in the 2012/13 
school year (with the number of study schools for each decile in parentheses): 2 (1), 3 (2), 

4 (1), 5 (5)> 6 (4), 7 (2), 8 (6), and 10 (3). 19 The 24 high schools were located across 
California, from the San Francisco Bay Area region down through Central California and 
throughout Southern California. The schools ranged in size from approximately 
1,500 students to approximately 3,700 students. 

Because the ERWC is a rigorous academic course meant to prepare students for college, 
certain subgroups of students were excluded from the impact analysis. 20 These include 
students who scored in the “Beginning” or “Early Intermediate” performance levels on the 


that matched non-ERWC students without replacement. The results of these analyses are discussed in 
the Sensitivity Analyses section of this chapter. 

18 School locales are based on definitions from the National Center for Education Statistics. “City, large 
territory” is defined as a territory inside an urbanized area and inside a principal city with a population 
of 250,000 or more. “City, mid-size territory” is defined as territory inside an urbanized area and inside a 
principal city with a population less than 250,000 and greater than 100,000. “City, small territory” is 
defined as territory inside an urbanized area and inside a principal city with a population less than 
100,000. “Rural, fringe census” is a census-defined rural territory that is less than or equal to 5 miles from 
an urbanized area, as well as rural territory that is less than or equal to 2.5 miles from an urban cluster. 
“Suburb, large territory” is defined as a territory outside a principal city and inside an urbanized area 
with a population of 250,000 or more. 

19 Academic performance data are based on the Academic Performance Index (API). The API is a single 
number, ranging from 200 to 1000, that reflects a school’s performance level, based on the results of 
statewide testing. The API is calculated by converting a student’s performance on statewide assessments, 
across multiple content areas, into points on the API scale. These points are then averaged across all 
students in the school and across all tests. Based on all schools’ API, the California Department of 
Education divides the population of schools into 10 deciles. 

20 The students that were excluded from the impact analysis were originally specified in the evaluation’s 
pre-analysis plan. 
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California English Language Development Test (CELDT) in grade 11, 21 students receiving 
special education services, and students who scored in the “Far Below Basic” performance 
level on the grade-11 ELA CST. 22 To be included in the impact analysis, students also 
needed to have non-missing data for the following variables: grade-11 ELA CST results, 
grade-11 English course enrollment, average grade-11 English course grade earned, 
ethnicity, gender, and EPT score. 

Also excluded from the analysis were students taught by teachers who were teaching the 
English course for the first time. This exclusion rule, which was specified in the 
pre-analysis plan, pertained to both ERWC and non-ERWC teachers. For instance, if a 
teacher was teaching English 4 and AP English Literature during the study period, and if 
the teacher had four years of prior experience teaching English 4 but no prior experience 
teaching AP English Literature, that teacher’s AP English Literature students were 
excluded from the analytic sample. There were eleven ERWC teachers whose students 
were excluded from the analytic sample because they had not previously taught the 
ERWC; there were six non-ERWC teachers who had students who were excluded because 
their students were enrolled in an English course that the teacher had not previously 
taught. 

ERWC and non-ERWC teachers, and the students they taught, were also excluded from 
the analysis if the teacher missed more than four consecutive weeks (that is, 

20 consecutive days) of school. The rationale for this exclusion criterion, which was 
specified in the pre-analysis plan, was that it is unlikely that a long-term substitute for an 
ERWC teacher would have had experience teaching the ERWC, which was one of the 
criteria for ERWC teachers’ inclusion in the study; non-ERWC teachers were excluded 
using this criterion so that both the treatment and the control groups would be equally 
affected. Two teachers (one ERWC teacher and one non-ERWC teacher) had missed more 
than four consecutive weeks of school during the year; these teachers and their students 
were excluded from the analysis. 

Finally, non-ERWC students who were taught by teachers who were also teaching the 
ERWC were excluded from the analysis. 23 The rationale for this decision rule, which was 


21 Students in kindergarten through grade 12 whose home language is not English are required by law to 
be assessed in English language proficiency. In California, the English language proficiency assessment is 
the CELDT. The CELDT performance levels are as follows, in order of increasing proficiency: 

“Beginning,” “Early Intermediate,” “Intermediate,” “Early Advanced,” and “Advanced.” 

22 The ELA CST performance levels are, in order of increasing proficiency, “Far Below Basic,” “Below 
Basic,” “Basic,” “Proficient,” and “Advanced.” Students who scored in the “Far Below Basic” performance 
level had scale scores between 150 and 258, inclusive. 

23 The WestEd research team had originally requested that ERWC teachers not teach grade-12 non- 
ERWC classes. However, this request was unable to be met in some instances. For instance, sometimes a 
teacher had a number of courses that he or she had been teaching for many years, or he or she was the 
only teacher experienced enough to teach a particular course. These circumstances, along with 
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stated in the pre-analysis plan, is that ERWC teachers may incorporate some ERWC 
strategies into their non-ERWC classes, and this “contamination,” or “bleeding,” may 
reduce the treatment/control contrast. If a teacher taught both the ERWC and a non- 
ERWC class, only the teacher’s ERWC classes were included in the study sample; as a 
result, the ERWC teachers and the non-ERWC teachers in the analysis form two mutually 
exclusive groups. Nine ERWC teachers also taught non-ERWC classes, and so the students 
in their non-ERWC classes were excluded from the analysis. 

The inclusion criteria used to arrive at the study sample impact the generalizability of the 
study findings. Elowever, in the Sensitivity Analyses section of this chapter, we report the 
results of a sensitivity analysis that includes all students who had non-missing data on the 
variables that were included in the matching analysis. This provides information on the 
robustness of the results from the main analysis. 

In total, 56 ERWC teachers and 58 non-ERWC teachers were included in the study sample. 
Thirty-two of the 56 ERWC teachers (57.1 percent) had received a master’s or Ph.D. 
degree, while 36 of the 58 non-ERWC teachers (62.1 percent) had received a master’s or 
Ph.D. degree. 24 In addition, the average numbers of years of total teaching experience were 
14.3 years for the ERWC teachers and 16.3 years for non-ERWC teachers. This information 
suggests that the group of ERWC teachers and the group of non-ERWC teachers were 
relatively similar along measurable characteristics. 

Data collected from the high school districts included 10,959 unique grade-12 students in 
the 24 study high schools in the 2013/14 school year (see figure 3.1). Based on test data 
received from the Educational Testing Service (ETS), which administered the EPT in the 
spring of 2014, a total of 7,631 students took the EPT. The student-level dataset received 
from ETS was then merged with the student-level data received from the school districts. 
Because the ETS dataset did not have the same student identification numbers that were 
used by the school districts, the two datasets were merged based on the following 
student-level variables: student first name, student last name, date of birth, and school of 
enrollment. Based on this “fuzzy” merge, the final merged dataset included 7,340 students. 
This final merged dataset included student demographic characteristics, student course 
enrollment, grade-11 course grades earned, standardized test scores, and EPT scores. 


scheduling constraints, made it impossible, in some cases, for a teacher to teach only the ERWC in 
grade 12. 

24 This information on advanced degrees is provided as evidence that there were not characteristics that 
solely distinguished ERWC teachers from non-ERWC teachers, which could have introduced a confound 
into the study design. 
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Figure 3.1. Consort Diagram for Students, 2013/14 


Grade-12 
enrollment in 
24 study high 
schools 


10,959 students 

9 school districts 
(Dataset from districts) 


Took the English 
Placement Test 
in spring 2014 


7,631 students 

(Dataset from ETS) 


"Fuzzy" merge to merge the 
dataset of the grade-12 
enrollment with the dataset of 
students who took the EPT. The 
fuzzy merge used the following 
student variables: first name, last 
name, date of birth, and school 
of enrollment. 


7,340 students 


Excluded 2,170 students from the 
final study sample, based on 
decisions made in the pre-analysis 
plan. 3 



5,170 students 

Final study 


3,309 students took the ERWC 

sample 


1,861 students took a 



non-ERWC English class 



Matching analysis 
conducted 


Notes: 

a. Students were excluded from the study sample if they met any of the following criteria: (1 ) taught 
by an English teacher who was teaching the course for the first time, (2) taught by an English teacher 
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who experienced an absence of at least 20 consecutive school days, (3) scored “Far Below Basic” 
on the grade-1 1 ELA CST, (4) received special education services, (5) scored “Beginning” or “Early 
Intermediate" on the CELDT, (6) were enrolled in a non-ERWC class that was taught by a teacher 
who also was teaching the ERWC, or (7) had missing data for any of the following variables: grade-1 1 
ELA CST, grade-1 1 English course enrollment, average grade-1 1 English course grade earned, 
ethnicity, gender, and EPT score. 

b. The numbers of non-ERWC students reported here are weighted numbers. Because there were four 
non-ERWC students matched to each ERWC student, a non-ERWC student received a weight of 0.25 
for every time he or she was matched to an ERWC student. For instance, if a non-ERWC student was 
matched to five different ERWC students, that non-ERWC student had a weight of 1 .25 in the final 
analytic sample (5 * 0.25 = 1 .25). The weighted number of non-ERWC students is the same as the 
unweighted number of ERWC students. 

Sources: English Placement Test (spring 2014) data and student records data collected from the nine 
school districts in the study sample. 

Table 3.1 provides the descriptive statistics of the study sample after certain students were 
excluded (as described earlier in this section), but before the matching was conducted. 
Overall, there were 3,309 ERWC students and 1,861 non-ERWC students in the study 
sample. 


Table 3.1. Descriptive Statistics of the Study Sample 


Student Characteristics 

Number 

Percentage 

Gender 



Female 

2,694 

52.1 1 

Male 

2,476 

47.89 

Ethnicity 



African American 

214 

4.14 

Asian 

1,381 

26.71 

Hispanic 

2,337 

45.20 

White 

1,238 

23.95 

Grade- 1 1 English course 



AP English Language 

1,236 

23.91 

Non-AP English course 

3,934 

76.09 

Grade- 12 English course 



ERWC 

3,309 

64.00 

English 4 

840 

16.25 

AP English Literature 

871 

16.85 

World literature 

121 

2.34 

Other (e.g., English literature, Mexican Chicano 

29 

0.56 

literature) 


Source: Student records data collected from the nine school districts in the study sample. 
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The Outcome Measure 


The EPT was the outcome measure used in the impact evaluation. The EPT is the 
standardized placement test given to students entering the CSU system who have not 
demonstrated English proficiency by any other measure, in order to determine students’ 
eligibility for placement into a credit-bearing college English course; students who are 
assessed as not proficient on the EPT are subject to remediation. 

The test, administered and scored by ETS, consists of reading-skills multiple-choice 
questions (30 minutes), composing-skills multiple-choice questions (30 minutes), and an 
essay (45 minutes). More specifically, with respect to reading skills, the EPT assesses 
students’ abilities to (among other skills) identify important ideas, understand direct 
statements, draw inferences and conclusions, detect underlying assumptions, recognize 
word meanings in context, and respond to tone and connotation (California State 
University, 2009). For composing skills, the EPT assesses students’ abilities to (among 
other skills) rewrite a sentence, choose the best version of a given sentence, understand 
sentence relationships within a passage, and select a sentence that provides the best 
support for a given topic. The essay portion of the test assesses the types of analytic 
writing skills required for in-class papers or examinations in college courses. The 
alignment of the EPT to the ELA content standards is provided in appendix A of California 
State University (2009). California State University (2009) contains further information 
about the EPT, including the design of the test, sample questions, and how it aligns to the 
content standards. 

The subsection scores for the reading- and composing-skills sections each have a 
minimum possible score of 120 and a maximum possible score of 180. The essay is scored 
on a 7-point scale (0-6). The scores on these three portions of the test are combined to 
arrive at a total scale score, which is the achievement measure used in this impact 
analysis. The total scale score is reported on a scale of 120-180. On average, the reliability 
of the EPT is .91 and the standard error of measurement is between 3 and 4 points on the 
score reporting scale of 120 to 180 (Educational Testing Service, 2011). 

All students who were participating in the study were asked to take the EPT on their high 
school campus in spring 2014. 25 ETS, which administers and scores the EPT each year for 
all students in the state of California who take the test, administered the test at each of 
the study high school campuses for the purposes of this evaluation. ETS then scored all of 
the tests and transferred (through secure file transfer protocol) the student-level results to 
WestEd. 


25 As per the CSU guidelines, students are only allowed to take the EPT one time. 
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Baseline Balance Testing 

Baseline balance testing is necessary to determine whether the sample of ERWC students 
is similar to the sample of non-ERWC students included in the analysis. This testing is 
conducted on the final analytic sample after the matching has been performed. Because 
the matching analysis was conducted with replacement, and because each ERWC student 
was matched to four non-ERWC students, each non-ERWC student in the analytic sample 
received a weight that was equivalent to the number of times that that non-ERWC student 
was used as a match, multiplied by 0.25 (Steiner & Cook, 2013). Each ERWC student 
received a weight of 1. Table 3.2 shows the balance that was achieved with respect to 
gender, ethnicity, and grade-11 AP English course enrollment. Table 3.3 provides the 
results of the baseline balance testing with respect to the baseline achievement measures: 
grade-11 ELA CST scale score and average grade-11 English grade earned. 

In table 3.2, the total number of ERWC students in the matched analysis is 3,309. This 
corresponds to the total number of ERWC students from the study sample. In other 
words, all ERWC students were matched to four non-ERWC students, and all ERWC 
students were included in the baseline balance tests — no ERWC students were dropped 
during the matching process as a result of having a “poor match” with a non-ERWC 
student. 26 With respect to the non-ERWC students, the total number of weighted 
non-ERWC students after the matching was conducted is 3,309. Among the 1,861 
(unweighted) non-ERWC students in the study sample, 27 1,632 non-ERWC students were 
matched to at least one ERWC student and therefore were included in the final analytic 
sample. In other words, 229 non-ERWC students were not matched to an ERWC student 
and hence were excluded from the analytic sample. 28 The results in table 3.2 indicate that 
the ERWC and non-ERWC students were exactly matched with respect to ethnicity and 
enrollment in AP English in grade 11. 


26 The reason that no ERWC students were dropped during the matching process is that each ERWC 
student was matched to the four closest non-ERWC students (“nearest neighbor” matching), regardless 
of how “close” the non-ERWC students were in terms of the Mahalanobis distance. 

27 Table 3.1 shows the unweighted number of non-ERWC students in the study sample, which is 
equivalent to 840 + 871 + 121 + 29 = 1,861. 

28 These 229 non-ERWC students were not matched to an ERWC student because they were not similar 
enough to any ERWC student. 
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Table 3.2. Baseline Balance for Gender, Ethnicity, and Grade-11 English Course Enrollment 


Student Characteristics 

ERWC Students 

Non-ERWC Students* 

Gender 



Female 

1,692 

1,692.25 

Male 

1,617 

1,616.75 

Ethnicity 



African American 

162 

162 

Asian 

755 

755 

Hispanic 

1,536 

1,536 

White 

856 

856 

Grade- 1 1 English course 



AP English 

442 

442 

Non-AP English course 

2,867 

2,867 

TOTAL OBSERVATIONS 

3,309 

3,309 


* The numbers of non-ERWC students reported are weighted numbers. 

Source: Student records data collected from the nine school districts in the study sample. 


The baseline equivalence with respect to the grade-11 ELA CST scale scores (shown in 
table 3.3) is particularly important because this is a baseline measure that is expected to be 
highly correlated with the EPT. As described in the CST blueprints provided on the 
California Department of Education website, the grade-11 ELA CST assesses content 
standards in reading and writing. 29 The content standards assessed in reading include 
word analysis/vocabulary (11 percent of the test), reading comprehension (25 percent), and 
literary response and analysis (23 percent); the content standards assessed in writing 
consist of written and oral language conventions (12 percent) and writing strategies 
(29 percent). As shown in table 3.3, between ERWC and non-ERWC students, there was a 
standardized mean difference of 0.015 in grade-11 ELA CST scale scores. As a result, 
baseline equivalence was achieved with respect to this pre-achievement variable. 30 


29 The grade-11 ELA CST blueprint can be found at http://www.cde.ca.gov/ta/tg/sr/blueprints.asp . This 
blueprint was retrieved on May 11, 2015. 

30 Ho, Imai, King, and Stuart (2007) recommend standard mean differences of, at most, 0.25 standard 
deviations on all variables. 
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Table 3.3. Baseline Balance for Grade-1 1 ELA CST Scale Score and Average Grade-1 1 
English Grade 




Non-ERWC 

Standardized 


ERWC Students 

Students 

Mean Difference 

Student Characteristics 

(n = 3309) 

(n = 3309)* 

(Hedges’ g) 

Grade- 1 1 ELA CST 
scale score 

356.2276 

(42.36429) 

356.8446 

(40.88805) 

0.015 

Average grade- 1 1 English 

2.672418 

2.701214 

0.030 

grade earned 

(0.9662466) 

(0.926831) 


* The number of non-ERWC students is a weighted number. 

Source: Student records data collected from the nine school districts in the study sample. 


Impact Results 

Table 3.4 displays the results from the OLS regression analysis that includes the matched 
ERWC and non-ERWC students. As described earlier in this chapter, each ERWC student 
received a weight of 1, and each non-ERWC student received a weight that was based on 
the number of times the student was matched. As a result of the weighting scheme, the 
regression model included all 3,309 ERWC students from the original study sample, and 
3,309 (weighted) non-ERWC students. 31 

The estimate for the ERWC enrollment variable in table 3.4 represents the impact 
estimate for the ERWC. The estimate (1.221) is positive and statistically significant at the 
1 percent level. This means that the ERWC was found to have a positive impact on student 
achievement, and that this impact was unlikely to have happened by chance. 


31 The weighted non-ERWC students were enrolled in the following English courses in grade 12 (with 
percentages in parentheses): English 4 (62.57 percent), AP English Literature (25.26 percent), world 
literature (10.85 percent), and other English course (1.32 percent). The “other English course” category 
consisted of courses such as English literature and Mexican Chicano literature. 
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Table 3.4. Ordinary Least Squares Regression Analysis Showing the Impact of the Expository 
Reading and Writing Course on English Placement Test Results 


Characteristic 

Estimate 

Robust 

Standard 

Error 

f-Statistic 

p-Value 

ERWC enrollment 

1 22 1 *** 

0.239 

5.12 

<.001 

Female 

-0.319 

0.240 

-1.33 

.184 

Asian 

.965 

0.843 

1.16 

.247 

Hispanic 

0.032 

0.813 

0.04 

.968 

White 

1.516* 

0.844 

1.80 

.073 

Grade- 1 1 ELA CST scale score 

0.136*** 

0.003 

46.28 

<.001 

Average grade- 1 1 English 
grade earned 

1 .365*** 

0.130 

10.50 

<.001 

Grade- 1 1 AP English enrollment 

2.533*** 

0.287 

8.81 

<.001 

Intercept 

87.743*** 

1.181 

74.27 

< .001 


* denotes statistical significance at the 10 percent level; ** denotes statistical significance at the 
5 percent level; *** denotes statistical significance at the 1 percent level. 

Note: With respect to ethnicity, African Americans were the omitted category as a result of being the 
first group alphabetically. 

Observations = 6,618 

Sources: English Placement Test (spring 2014) data and student records data collected from the nine 
school districts in the study sample. 

The estimated effect size of the ERWC is 0.13 (table 3.5). In calculating the effect size, the 
adjusted mean difference from the regression model was used (from table 3.4), as 
suggested in Lipsey et al. (2012). 

Table 3.5. Estimated Effect Size of the Expository Reading and Writing Course on the English 
Placement Test 


ERWC 
Group 
Mean on 
EPT 

Number of 
ERWC 
Students 

Non-ERWC 
Group 
Mean on 
EPT 

Number of 
Non-ERWC 
Students 0 

Pooled 

Within- 

Group 

Standard 

Deviation 

Adjusted 

Mean 

Difference 13 

Estimated 
Effect Size 

141.929 

3,309 

140.831 

3,309 

9.1598 

1.221 

0.13 


Notes: 

a. This represents the weighted number of students, based on the number of times each non-ERWC 
student was matched to an ERWC student. 

b. The adjusted mean difference is the estimated regression coefficient for ERWC from table 3.4, as 
opposed to the difference in English Placement Test means between the two groups. Using the 
regression coefficient controls for the covariates that were included in the regression model. 

Sources: English Placement Test (spring 2014) data and student records data collected from the nine 
school districts in the study sample. 
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The Effect Size in Context 


To assist the reader in understanding the relative magnitude of the estimated effect size, 
as shown in table 3.5, this section provides information and previous guidance from 
published reports. Numerous papers have stressed the importance of considering key 
elements of an evaluation in order to put the estimated effect size into context. As noted 
in Hill, Bloom, Black, and Lipsey (2008), when evaluating the relative magnitude of an 
effect size, one should take into account the outcome measure used in the evaluation, the 
population being studied (e.g., the grade level), and the nature of the intervention being 
evaluated. 

Outcome Measure 

Hill et al. (2008) have shown that the choice of the outcome measure will affect the 
magnitude of the effect size. They categorize outcome measures into three groups: 
standardized tests that cover broad subject matter (such as the SAT9 composite reading 
test), standardized tests that focus on a narrow topic (such as the SAT9 vocabulary test), 
and specialized tests developed specifically for an intervention (such as a reading- 
comprehension measure developed by a researcher for text similar to that used in the 
intervention). 

Hill et al. (2008) found that broad standardized tests tend to have the smallest average 
effect sizes (mean = 0.07 in the elementary grades), with effect sizes from narrowly 
focused standardized tests being larger (mean = 0.23 in the elementary grades) and 
specialized tests having the largest effect sizes (mean = 0.44 in the elementary grades). 32 
“These findings,” they note, “raise important issues about how the test used to measure 
the effectiveness of an educational intervention might influence the results obtained” 

(p. 176). Lipsey et al. (2012) present similar findings, showing that, among high school 
studies, evaluations that used a specialized topic or test had an average effect size of 0.34, 
while evaluations that used a narrowly focused standardized test had an average effect size 
of 0.03. They found only one evaluation at the high school level that used a standardized 
test with a broad scope, and the results from that study were not provided; however, one 
would expect that effect sizes from evaluations that use broad standardized tests would be 
smaller, on average, than evaluations that use narrowly focused standardized tests. The 
EPT, which was the assessment used for this impact evaluation, would be considered a 
broad standardized test. 


32 Hill et al. (2008) do not break out the average effect sizes by type of outcome measure at the high 
school level. For this reason, we report the results that they have provided at the elementary school level. 
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Population Studied 

Hill et al. (2008) also stressed the importance of comparing interventions that target 
similar grade levels. This is due to their findings that middle school studies tended to have 
the largest effect sizes (mean = 0.51), with elementary school studies having the second- 
largest effect sizes (mean = 0.33) and high school studies having the lowest (mean = 0.27). 

Intervention Evaluated 

With respect to the nature of the intervention being evaluated, Lipsey et al. (2012) 
distinguish among five types of intervention, each of which will likely have a different 
average effect size: (1) instructional format interventions, such as cooperative learning, 
peer-assisted learning, and simulation games; (2) teaching technique interventions — that 
is, specific pedagogical techniques or simple augmentations to instruction; 

(3) instructional component or skill training, such as phonological awareness training and 
computer-assisted tutoring on specific content; (4) curriculum or broad instructional 
program interventions, such as a science or math curriculum; and (5) whole-school 
program interventions, such as comprehensive school reform initiatives. The ERWC 
would be considered a curriculum or broad instructional program. When disaggregating 
average effect sizes by the type of intervention, Lipsey et al. (2012) report the following 
average effect sizes across the five types (the average effect size for each type is in 
parentheses): instructional format (0.21), teaching technique (0.35), instructional 
component or skill training (0.36), curriculum or broad instructional program (0.13), and 
whole-school program (0.11). These results suggest the importance of comparing similar 
types of interventions to each other. In other words, the effect size of the ERWC should be 
compared to the effect size of other curriculum or broad instructional program 
interventions. 

Sensitivity Analyses 

A number of sensitivity analyses were performed to assess the robustness of the main 
impact estimate that is presented in table 3.4. These sensitivity analyses varied by the 
manner in which the matching was conducted (Sensitivity Analyses 1, 2, and 3) and the 
sample that was used in the analysis (Sensitivity Analysis 4). All sensitivity analyses 
included a matching analysis and a regression analysis that included the matched ERWC 
and non-ERWC students. 

Sensitivity Analysis 1 used propensity score matching rather than matching on the 
Mahalanobis distance. All other matching decisions were the same as those used in the 
main impact estimate: Each ERWC student was matched to the four nearest non-ERWC 
students, each non-ERWC student was weighted 0.25 for every time the non-ERWC 
student was matched to an ERWC student, and matching was conducted with 
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replacement. In this sensitivity analysis, 3,309 ERWC students and 3,309 (weighted) 
non-ERWC students were included in the regression analysis. 

Sensitivity Analysis 2 performed a one-to-one match, with each ERWC student matched to 
one non-ERWC student; as a result, each non-ERWC student was weighted as 1 unit for 
every time he or she was used as a match for an ERWC student. All other matching 
decisions were the same as used in the main impact estimate (the Mahalanobis distance 
metric was used, and matching was conducted with replacement). This sensitivity analysis 
included 3,309 ERWC students and 3,309 (weighted) non-ERWC students. 

Sensitivity Analysis 3 employed matching without replacement. Because there were fewer 
non-ERWC students than ERWC students in the final study sample (1,861 versus 3,309; see 
figure 3.1), one-to-one matching was performed — if each ERWC student needed to be 
matched to four non-ERWC students while at the same time matching without 
replacement, very few ERWC students could have been included in the analysis. Matching 
without replacement, therefore, was similar to if one were to perform matching on the 
non-ERWC students (as opposed to matching on the ERWC students, as was the case in 
the main impact analysis), since the number of non-ERWC students was the limiting 
factor in the matching process. In addition, a caliper restriction was necessary, whereby 
non-ERWC students needed to be within a certain caliper distance from the ERWC 
student that he or she was matched to, in order to achieve baseline equivalence with 
respect to the grade-11 ELA CST scores; a caliper of 0.05 was used. Furthermore, propensity 
scores, rather than the Mahalanobis distance, were used to conduct the matching, because 
the Match It command in the R statistical software did not allow caliper requirements with 
the Mahalanobis-distance metric. 

Sensitivity Analysis 4 included all students in the dataset who had non-missing data for 
the variables that were included in the matching analysis. More specifically: Certain 
students were excluded from the main analysis, based on the pre-analysis plan. For 
instance, excluded from the main impact analysis were students receiving special 
education services; students who scored “Far Below Basic” on the grade-11 ELA CST; 
English language learners who scored either “Beginning” or “Early Intermediate” on the 
CELDT; students taught by teachers who were teaching the English course for the first 
time; and students of teachers who had had extended absences. In Sensitivity Analysis 4, 
these students were included in the analysis as long as they did not have missing data with 
respect to any of the following variables: EPT score, grade-11 ELA CST score, average 
grade-11 English grade earned, grade-11 English course enrollment, gender, and ethnicity. 
The same matching method was used in Sensitivity Analysis 4 as was used in the main 
impact estimate. 
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Table 3.6. Sensitivity Analyses 



Main 

Impact 

Estimate 

Sensitivity 
Analysis 1 
(Propensity 
Score) 

Sensitivity 
Analysis 2 
(One-to- 
One Match) 

Sensitivity 
Analysis 3 
(Matching 
Without 
Replacement) 

Sensitivity 
Analysis 4 
(Full 

Sample) 

Regression 

coefficient 

1 22 1 *** 

1 245 *** 

1.164*** 

1.126*** 

1 .283*** 

Standard error a 

0.239 

0.223 

0.281 

0.225 

0.188 

Effect size 

0.13 

0.13 

0.13 

0.1 1 

0.14 

Total sample 
size b 

6,618 

6,618 

6,618 

3,002 

8,132 

ELA CST 
difference c 

0.015 

0.109 

0.009 

0.024 

0.01 1 


* denotes statistical significance at the 10 percent level; ** denotes statistical significance at the 
5 percent level; *** denotes statistical significance at the 1 percent level. 

Notes: 

a. Robust standard errors are reported when the sensitivity analysis involved one-to-many matching. 

b. When one-to-many matching was performed, each non-ERWC student was weighted based on 
the number of times that the non-ERWC student was matched to an ERWC student. ERWC students 
are always weighted to be equal to one. 

c. The ELA CST difference represents the standardized mean difference in grade-1 1 ELA CST scores 
between the ERWC students and the non-ERWC students. Because the grade-1 1 ELA CST score 
represented the pre-achievement variable of interest in the matching analysis, the other matching 
variables are not reported. 

Sources: English Placement Test (spring 2014) data and student records data collected from the nine 
school districts in the study sample. 


As shown in table 3.6, all of the sensitivity analyses produced results that were similar to 
the main impact estimate: all resulted in impact estimates that were positive and 
statistically significant, with effect sizes ranging from 0.11 to 0.14. 33 In particular, the 
similarity of the results for Sensitivity Analyses 3 and 4 to the main findings provides 
considerable assurance that the main impact result is robust to alternative specifications. 


33 While the research team conducted additional sensitivity analyses, we only report the most extreme 
sensitivity analyses. For instance, the research team conducted a sensitivity analysis in which each 
ERWC student was matched to two non-ERWC students, but since Sensitivity Analysis 2 used only one 
non-ERWC student match and the main impact analysis used four matches, we do not present the 
results of intermediate numbers of matches. As another example, many potential choices could have 
been made with respect to the study sample, such as whether to include students taught by 
inexperienced teachers, students taught by teachers with extended absences, students scoring “Far 
Below Basic” on the grade-11 ELA CST, or any combinations thereof. Since the main impact analysis 
excludes all of these students while Sensitivity Analysis 4 includes all of these students, we do not 
provide results for any of the combinations that lie within the two ends of the spectrum. All other 
sensitivity analyses that were conducted but not reported resulted in positive and statistically significant 
results. 


30 



Chapter 4. Assessing Fidelity of 
Implementation of the Expository 
Reading and Writing Course 
During the Study Period 


The Expository Reading and Writing Course (ERWC) consists of three key components: 

(1) the curriculum, (2) the professional learning, and (3) the curriculum materials. In order 
to collect data on and assess whether each of the three key components was implemented 
with fidelity, data-collection instruments were developed by the evaluation team in 
collaboration with the developers of the ERWC. These data-collection instruments were 
distributed to study participants, to be completed during the study period. The 
instruments gathered data on fidelity of implementation and also provided respondents 
with opportunities to share their perspectives regarding the course and their experiences 
with it. 34 The primary data-collection instruments were implementation feedback charts, 
coaching logs, and professional learning community (PLC) logs. 35 The implementation 
feedback charts documented which activities, within each of the modules, each teacher 
covered; the coaching logs collected information about the coaching sessions; and the PLC 
logs collected information about the PLC meetings. These instruments are described in 
the following sections, as is the methodology for calculating fidelity of implementation for 
each component. This chapter also presents the results of the evaluation of 
implementation fidelity. 


34 Chapter 5 in this report provides an exploratory summary of the qualitative findings from these data- 
collection instruments, based on open-ended responses from study participants. 

35 A fourth data instrument that informed the qualitative findings in chapter 5 was an end-of-year survey 
completed by ERWC teachers and coaches during the summer after the implementation year. Findings 
from these surveys are included in chapter 5, which presents exploratory data findings based on open- 
ended responses from ERWC teachers and coaches. 
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Data-Collection Instruments for Assessing Fidelity 
of Implementation 

Implementation Feedback Charts 

Implementation feedback charts served as the primary tool for capturing the ERWC 
teachers’ day-to-day curriculum implementation. A copy of the hard-copy version of the 
implementation feedback chart for module 1 is provided as an example in appendix B. 
While almost all teachers completed the charts online, using SurveyMonkey, teachers also 
had the option to either (1) write their responses on a paper copy of the chart and give it to 
the school-site contact person, who would then mail it to the WestEd evaluation team, or 
(2) complete the electronic version in Microsoft Word and email it to the WestEd 
evaluation team. Through SurveyMonkey, WestEd provided each teacher with a teacher- 
specific survey link for each module, and whenever a teacher filled out a survey/chart 
using the link, all responses were captured in SurveyMonkey. All teachers were asked to 
complete and submit the survey for each module within two weeks of completing the 
module. 

As described in chapter 2 of this report, the ERWC curriculum consists of 12 modules, each 
focusing on a particular topic, such as juvenile justice, racial profiling, or bullying. The 
structure of each module follows the Assignment Template, shown in table 2.2 in chapter 
2. For explanatory purposes, in this evaluation, the Assignment Template can be thought 
of as consisting of three different levels. The first level denotes the overarching domains of 
the Assignment Template: Reading Rhetorically, Connecting Reading to Writing, and 
Writing Rhetorically. Within each domain are the strands, or second-level subsections. So, 
for example, within the Reading Rhetorically domain are the following strands: 

Prereading, Reading, and Postreading; within the Connecting Reading to Writing domain 
is the Discovering What You Think strand; and within the Writing Rhetorically domain 
are the Entering the Conversation and Revising and Editing strands. 

As noted earlier, within each strand are elements, or skill areas, that need to be taught and 
learned if students are to be able to master the given strand and, ultimately, its larger 
domain. For instance, within the Prereading strand are the following elements: Getting 
Ready to Read, Exploring Key Concepts, Surveying the Text, Making Predictions and 
Asking Questions, and Understanding Key Vocabulary. Associated with each of these 
elements are activities that help students develop the skills in the given element. For 
instance, an activity that asks students to reflect and discuss a particular topic may align 
to the Getting Ready to Read element . 36 


36 It is important to note that not all modules may contain all of the elements and/or their activities. For 
instance, a particular module may not contain an activity that relates to the element Making Predictions 
and Asking Questions. 
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Data from the implementation feedback charts were used to understand which modules 
and, within those modules, which activities the teachers taught in their classrooms. 37 For 
each activity in each module, teachers were asked to indicate: (1) whether the teacher 
completed the activity; (2) if the teacher did not complete the activity, a specific reason 
that the activity was not completed; (3) whether the activity was modified; (4) the amount 
of time spent on the activity; (5) the level of depth achieved for the activity; and (6) any 
successes, challenges, modifications, or variations related to the activity (this was an open- 
ended response). 

Teachers were considered to have taught the activity if they marked on the 
implementation feedback chart that they had completed the activity. Teachers who had 
not completed a particular activity were asked to indicate a reason, drawing from the 
following set of choices: (a) “Students had already mastered this knowledge/skill,” 

(b) “I substituted this activity with another activity that was better suited for my students’ 
needs,” (c) “I combined this activity with another activity in this module,” (d) “It was 
noted as an optional activity,” (e) “Our class did not have enough time to work on this 
activity,” or (f) “Other (please note what it is in the “Notes” column).” For purposes of 
calculating fidelity of implementation, a teacher was considered to have covered the 
activity if he or she marked “A,” “C,” or “D” as the reason for not having completed the 
activity. In other words, teachers were considered to have completed an activity if they 
marked “Completed” for the activity or if they marked “A,” “C,” or “D” as the reason for not 
having completed the activity. 

For a teacher to be considered to have taught a module with adequate fidelity, the teacher 
needed to have completed at least one activity in each of the module’s six strands 
(Prereading, Reading, Postreading, Discovering What You Think, Entering the 
Conversation, and Revising and Editing). In many cases, a teacher may have completed 
many activities throughout the entire module, but may not have completed an activity in 
one particular strand; in such a case, the teacher would not be considered to have taught 
the module with adequate fidelity. 38 To be considered to have taught the entire 
curriculum with fidelity, teachers needed to have taught at least eight modules with 
adequate fidelity over the course of the year. 


37 Discussions with ERWC teachers prior to the implementation year confirmed that when teachers 
teach multiple sections of the ERWC, all sections tend to move at the same pace and incorporate the 
same activities. This enabled each ERWC teacher to complete just one implementation feedback chart, 
rather than needing to fill out one chart for each section of the ERWC taught. The latter option would 
have been too burdensome for the study participants, given their many teaching responsibilities. 

38 Teachers in the study were not told how the implementation-fidelity calculation was to be made for 
this study. In other words, they were not told that they needed to complete at least one activity in each 
of the six strands to be considered to have taught the module. The rationale for not disclosing this 
information to the teachers was that teachers were allowed to choose for themselves the best activities 
to teach, based on their students’ needs. 
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Coaching Logs 

Coaches were expected to meet with study teachers twice per semester (four times per 
year). After each visit, coaches filled out a coaching log, which was a data-collection 
instrument created by the evaluation team to document both attendance at the coaching 
meetings and what happened at the meetings. A template of the coaching log is provided 
in appendix B. In addition to documenting participation in a coaching meeting, the 
coaching log contains five primary questions that coaches answered, based on their 
observations of classes and on their meetings with the teachers: (1) “What is going well?” 
(2) “What are some challenges that arose in the lesson?” (3) “What steps are being taken 
to address these challenges?” (4) During the observed session, what was your sense of the 
students’ engagement? To what do you attribute this level of engagement?” and (5) “What 
did the lesson do to help students become better readers and writers?” The logs also 
provided space for coaches to write notes and to identify next steps. 

Information from the coaching logs provides insights into the coaches’ perspectives on the 
implementation of the ERWC by the participating teachers. The majority of these logs 
captured coaches’ observations of classroom practice, as well as summaries of their 
conversations with individual teachers before and/or after the classroom visits. After 
completing each log, the coaches submitted a copy of the log to the Fresno County Office 
of Education, which then forwarded the logs to the WestEd evaluation team. 

Professional Learning Community Logs 

ERWC teachers were part of an active community of learners at the school site or district- 
level site through participation in PLCs. In order to document attendance in a PLC as well 
as what was discussed in the PLC meetings, teachers filled out a PLC log after each 
meeting. The PLC log template is provided in appendix B. In addition to documenting 
teacher attendance in these PLC meetings, the log asked teachers to respond to the 
following questions: (1) “What topics were covered during the ERWC PLC meeting?” 

(2) “What are some challenges that teachers are encountering?” and (3) “What successes 
are teachers experiencing with the ERWC curriculum?” Teachers were also asked to 
provide next steps. The lead ERWC teacher within each PLC was responsible for sending 
the completed PLC log to WestEd at the end of each month. 

Evaluating the Fidelity of Implementation of the ERWC 
Components 

WestEd collected data throughout the implementation year to determine the extent to 
which each of the three ERWC components (curriculum, professional learning, and 
curriculum materials) was being implemented with fidelity by each study teacher. The 
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proportions of teachers implementing each component with fidelity were then calculated 
to arrive at a component -level implementation-fidelity score among all study teachers. 

The scoring rubric for each of the three components is provided in a chart in appendix C 
and described in this section. 

With respect to the ERWC curriculum and the teaching of that curriculum, the number of 
modules that a teacher taught with adequate fidelity represented the teacher’s fidelity 
score. Because the course has 12 modules, a teacher’s fidelity score for this component 
could range from o to 12. As previously described, a teacher needed to have completed at 
least 8 modules with adequate fidelity in order to be considered to have taught the entire 
curriculum with fidelity. In order to be considered to have taught any single module with 
adequate fidelity, a teacher needed to have completed at least one activity in each of that 
module’s six strands. 

As shown in table 4.1, of the 56 teachers who participated in the study, 10 (17.9 percent) 
taught at least 8 modules with adequate fidelity. This percentage was low due to the 
stringent criterion established for determining whether a teacher taught a given module 
with fidelity. As previously discussed, for a teacher to be considered as having taught any 
module with fidelity, he or she needed to complete at least one activity in each of the six 
strands of the module. The ERWC teachers who participated in the study were not 
instructed to complete at least one activity in each of the module’s six strands, nor were 
they told that they would need to do this in order to be considered as having taught the 
module with fidelity. 

If, instead, the criterion had been defined as teaching at least five activities in a given 
module, irrespective of the strand — an approach that would better indicate the number 
of teachers who completed 8 different modules with their students over the course of the 
year — 35 teachers (62.5 percent) would have been considered to have taught at least 
8 modules. But even if the evaluation had used this lower threshold, 21 teachers still did 
not attempt to teach at least 8 modules over the course of the year. Communications with 
these teachers revealed a number of reasons why they had not attempted to teach 
8 modules over the school year. For example, one teacher explained that his PLC had 
made the decision for its participants to teach only 7 modules over the course of the year 
because the PLC members felt that it was very important to supplement each module with 
additional texts. The PLC members discussed these additional texts, along with additional 
research and writing practice, with the goal of teaching what was best for their students. 
Other teachers noted that some of their students, upon entering the class, had not had the 
necessary skills to handle the ERWC’s rigorous demands; these foundational skills — 
which, one teacher noted, are now beginning to be introduced at lower grade levels to 
remedy this issue — would have allowed the class to move at a faster pace throughout the 
year. As a last example, some teachers noted that some district requirements (such as 
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writing and reading assessments) took up class time throughout the year that could 
otherwise have been dedicated to the ERWC modules. 

As previously discussed, the ERWC’s professional-learning component consisted of three 
individual subcomponents: a two-day summer professional-learning session, the PLC 
meetings, and the coaching sessions. Fidelity scores for each of these three individual 
subcomponents were assessed for each ERWC teacher and then summed for that teacher, 
to arrive at an individual professional-learning fidelity score for every teacher. 

Teacher attendance at the two-day summer session was documented through sign-in 
sheets, with teachers required to sign in twice each day: once in the morning and once in 
the afternoon. In scoring teachers for this subcomponent of the professional learning, 
teachers were given a fidelity score of o if they did not attend any summer professional- 
learning days, a score of 1 if they attended one day, and a score of 2 if they attended 
two days. As shown in table 4.1, based on the sign-in sheets, 54 of 56 teachers 
(96.4 percent) attended both days of the two-day summer session. 

Teacher attendance in the PLC meetings was documented through the PLC logs. Teachers 
were given a fidelity score of o if, over the course of the year, they attended between zero 
and two PLC meetings. Attending three or four meetings earned a fidelity score of 1, 
attending five or six meetings earned a score of 2, attending seven or eight meetings 
earned a 3, and attending nine or more meetings earned a 4. A teacher was considered to 
have implemented this subcomponent of professional learning with fidelity if he or she 
received a score of 4, having attended at least nine PLC meetings. Of the 56 teachers, 

41 (73.2 percent) received a score of 4 for this professional-learning subcomponent. 

The fidelity score for the coaching subcomponent of professional learning was equal to the 
number of coaching sessions in which a teacher participated, with teachers able to receive 
a maximum score of 4 in this category. In other words, teachers received a fidelity score of 
o if they did not participate in any coaching sessions during the school year. They received 
a 1 if they attended one coaching session, a 2 if they attended two sessions, and so on. Of 
the 56 teachers, 53 teachers (94.6 percent) attended at least four coaching sessions. 

As noted earlier, to arrive at each teacher’s overall fidelity score for the ERWC’s 
professional-learning component, the fidelity scores for the three professional-learning 
subcomponents were summed. A teacher would need a score of at least 9 in order to have 
met the threshold for professional-learning implementation fidelity. In the evaluation, 

46 of 56 teachers (82.1 percent) received a fidelity score of at least 9 for this ERWC 
component. This percentage exceeds the pre-specified threshold of having at least 
80 percent of the teachers receiving a score of at least 9. Thus, the professional-learning 
component of the ERWC was deemed to have been implemented with fidelity. 

With respect to the curriculum materials, teachers received a fidelity score of o if they had 
not received all of the materials, and teachers received a fidelity score of 1 if they had 
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received all of the materials. From communications with all of the teachers throughout 
the implementation year (through emails and phone calls), the study team learned that all 
56 study teachers received all of the curriculum materials and, thus, received a fidelity 
score of 1 for this component. 

Table 4.1. Implementation Fidelity Results 


ERWC Component 

Number of 
teachers 
who 

implemente 
d with 
adequate 
fidelity 

Total 

number of 
teachers 

Percentage 
of teachers 
who 

implemented 

with 

adequate 

fidelity 

Prespecified 
percentage 
of teachers 
needed to 
meet the 
fidelity 
threshold 
for the 
component 

Met 

threshold? 

Curriculum taught with 
Fidelity 

10 

56 

17.9 

80 

No 

Professional Learning 

46 

56 

82.1 

80 

Yes 

Summer Professional 
Learning 

54 

56 

96.4 



Professional Learning 
Community 

41 

56 

73.2 



Coaching 

53 

56 

94.6 



Curriculum Materials 

56 

56 

100.0 

95 

Yes 


Source: Survey data collected from study participants. 


While it is important to know how many modules each teacher taught with fidelity 
throughout the school year, it is also important to understand which activities were 
taught. This provides information about the percentages of activities completed in each of 
the three categories (domains, strands, and elements) within a module. This information 
is helpful in understanding, for instance, whether teachers were more likely to teach 
activities in the Reading Rhetorically domain versus the Writing Rhetorically domain, and, 
within the Writing Rhetorically domain, whether teachers were more likely to complete 
activities in the Entering the Conversation strand as compared to activities in the Revising 
and Editing strand. 

To briefly reference each element in the Assignment Template (table 2.2), table 4.2 assigns 
a number to each element. For instance, in table 4.2, the first element, Getting Ready to 
Read, is referred to as element number 1; the second element, Exploring Key Concepts, is 
referred to as element number 2; and so on. Referencing elements using numbers is useful 
for the graphical illustration of the completion rates of the different elements (figure 4.1). 
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Table 4.2. The Element Numbers 


Element 

Number 

Domain/Strand 

Element 

1 

Reading Rhetorically: Prereading 

Getting Ready to Read 

2 

Reading Rhetorically: Prereading 

Exploring Key Concepts 

3 

Reading Rhetorically: Prereading 

Surveying the Text 

4 

Reading Rhetorically: Prereading 

Making Predictions and Asking Questions 

5 

Reading Rhetorically: Prereading 

Understanding Key Vocabulary 

6 

Reading Rhetorically: Reading 

Reading for Understanding 

7 

Reading Rhetorically: Reading 

Considering the Structure of the Text 

8 

Reading Rhetorically: Reading 

Noticing Language 

9 

Reading Rhetorically: Reading 

Annotating and Questioning the Text 

10 

Reading Rhetorically: Reading 

Analyzing Stylistic Choices 

1 1 

Reading Rhetorically: Postreading 

Summarizing and Responding 

12 

Reading Rhetorically: Postreading 

Thinking Critically 

13 

Reading Rhetorically: Postreading 

Reflecting on Your Reading Process 

14 

Connecting Reading to Writing: 
Discovering What You Think 

Considering the Writing Task 

15 

Connecting Reading to Writing: 
Discovering What You Think 

Taking a Stance 

16 

Connecting Reading to Writing: 
Discovering What You Think 

Gathering Evidence to Support Your 
Claims 

17 

Connecting Reading to Writing: 
Discovering What You Think 

Getting Ready to Write 

18 

Writing Rhetorically: Entering the 
Conversation 

Composing a Draft 

19 

Writing Rhetorically: Entering the 
Conversation 

Considering Structure 

20 

Writing Rhetorically: Entering the 
Conversation 

Using the Words of Others (and Avoiding 
Plagiarism) 

21 

Writing Rhetorically: Entering the 
Conversation 

Negotiating Voices 

22 

Writing Rhetorically: Revising and Editing 

Revising Rhetorically 

23 

Writing Rhetorically: Revising and Editing 

Considering Stylistic Choices 

24 

Writing Rhetorically: Revising and Editing 

Editing the Draft 

25 

Writing Rhetorically: Revising and Editing 

Responding to Feedback 

26 

Writing Rhetorically: Revising and Editing 

Reflecting on Your Writing Process 


Source: The Expository Reading and Writing Course curriculum. 
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Table 4.3. Completion Rates of Activities by Domain, Strand, and Element 


Domain, Strand, 
or Element 

Name 

Percentage 

Completed ab ' c 

Domain 

Reading Rhetorically 

78 

Strand 

Prereading 

84 

Element 

Getting Ready to Read 

91 

Element 

Exploring Key Concepts 

83 

Element 

Surveying the Text 

87 

Element 

Making Predictions and Asking Questions 

83 

Element 

Understanding Key Vocabulary 

78 

Strand 

Reading 

75 

Element 

Reading for Understanding 

82 

Element 

Considering the Structure of the Text 

77 

Element 

Noticing Language 

64 

Element 

Annotating and Questioning the Text 

83 

Element 

Analyzing Stylistic Choices 

69 

Strand 

Postreading 

72 

Element 

Summarizing and Responding 

77 

Element 

Thinking Critically 

76 

Element 

Reflecting on Your Reading Process 

63 

Domain 

Connecting Reading to Writing 

74 

Strand 

Discovering What You Think 

74 

Element 

Considering the Writing Task 

78 

Element 

Taking a Stance 

72 

Element 

Gathering Evidence to Support Your Claims 

70 

Element 

Getting Ready to Write 

77 

Domain 

Writing Rhetorically 

63 

Strand 

Entering the Conversation 

70 

Element 

Composing a Draft 

81 

Element 

Considering Structure 

74 

Element 

Using the Words of Others (and Avoiding 
Plagiarism) 

69 

Element 

Negotiating Voices 

48 

Strand 

Revising and Editing 

57 

Element 

Revising Rhetorically 

61 

Element 

Considering Stylistic Choices 

50 

Element 

Editing the Draft 

64 
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Domain, Strand, 
or Element 


Name 


Percentage 

Completed ab ' c 


Element 

Responding to Feedback 

51 

Element 

Reflecting on Your Writing Process 

54 


Notes: 

a. The domain averages are based on the average completion rates of all elements in each domain 
for all 12 modules. For instance, the average for the Reading Rhetorically domain is calculated as the 
average completion rate of elements 1 through 13 among all 12 modules. 

b. The strand averages are based on the average completion rates of all elements in each strand for 
all 1 2 modules. For instance, the average for the Prereading strand is calculated as the average 
completion rate of elements 1 through 5 among all 12 modules. 

c. The element averages are based on the average completion rates for that particular element for 
all 1 2 modules, with each module being equally weighted. 

Source: Survey data collected from study participants. 

Table 4.3 presents the percentages of activities completed at each of the three levels 
within a module: domain, strand, and element. It shows that, at the domain level, teachers 
were most likely to complete activities in the Reading Rhetorically domain and least likely 
to complete activities in the Writing Rhetorically domain; teachers completed 78 percent 
of Reading Rhetorically activities, 74 percent of Connecting Reading to Writing activities, 
and 63 percent of Writing Rhetorically activities. There is also a pattern of decline as 
teachers progress through the strands. For instance, within the Reading Rhetorically 
domain, the Prereading strand had the highest rates of completion (84 percent), then 
Reading (75 percent), and then Postreading (72 percent). Similarly, within the Writing 
Rhetorically domain, the Entering the Conversation strand had a completion rate of 
70 percent, while Revising and Editing had a completion rate of 57 percent. 

These numbers show that teachers tended to complete a higher proportion of the 
activities at the beginning of the module, and then, most likely due to feeling pressured to 
complete the module on schedule, tended to complete a lower proportion of activities 
later in the module. This also indicates that teachers tended to spend less time on the 
writing portion of the module and, in particular, on the revising and editing of students’ 
work. As one ERWC teacher noted in a survey, “I need to be more deliberate before 
teaching each module to choose specific pieces. I tend to do more at the beginning and 
then hurry at the end.” Pacing challenges that ERWC teachers encountered are explored 
further in chapter 5. 

Figure 4.1 provides a visual representation of the percentage of activities completed for 
each element number (refer to table 4.2 to identify the element associated with each 
element number). This visual representation of the completion rates by element also 
shows that teachers tended to complete a lower proportion of activities as they progressed 
through the module. 
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Figure 4.1. Percentage of Activities Completed for Each Element 



Note: The green line pertains to the Reading Rhetorically activities. The yellow line pertains to the 
Connecting Reading to Writing activities. The blue line pertains to the Writing Rhetorically activities. 
Source: Survey data collected from study participants. 
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Chapter 5. Qualitative 
Survey Findings 


During the implementation year for this study of the Expository Reading and Writing 
Course (ERWC), the research team distributed a number of data-collection instruments to 
study participants, with the aim of gathering both implementation information and 
participant feedback about the course. The instruments, described in the previous 
chapter, were implementation feedback charts, coaching logs, and professional learning 
community (PLC) logs; an end-of-year survey was also distributed. The qualitative 
open-ended responses gathered using these instruments serve as the basis for the findings 
presented in this chapter, which first cover the ERWC curriculum and then focus on two 
aspects of the ERWC professional learning: coaching and the PLCs. Information related to 
the curriculum was derived primarily from the implementation feedback charts, while 
information about the coaching and PLCs came primarily from the logs related to these 
aspects. 

The findings in this chapter provide context for the study’s quantitative findings and are 
exploratory in nature, summarizing feedback received on the various components of the 
curriculum and providing context about how the course was implemented. They are 
intended to serve as a starting point in considering potential future topics of exploration 
with respect to the ERWC. They can also serve as valuable input to help the course 
developers better understand the implementation of the curriculum during the study 
period. 

Analysis of the open-ended responses in the data-collection instruments began with the 
research team creating initial codes based on common themes that emerged from the 
data. This process involved creating many codes using open coding techniques, with each 
data source being coded by at least two researchers . 39 To ensure a uniform coding process, 
the research team met regularly to discuss the relevance and importance of each code, and 
refined and updated codes as necessary. Themes that emerged across data-collection 
instruments were also compared, in order to triangulate experiences with the ERWC from 
different points of views. The research team then developed the general findings for this 
chapter, based on the salient themes. 


39 Atlas. ti qualitative data analysis software was used to help examine the open-ended text responses 
from the implementation feedback charts. This software was used for this particular data source because 
of the large amounts of open-ended information in this survey instrument. 
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ERWC Curriculum 


ERWC teachers used the implementation feedback charts to provide information and 
feedback about the activities that they taught in each module, as well as factors that 
helped or hindered the implementation of the course. This section summarizes major 
themes emerging from the qualitative information that they provided. 

Finding 1: Teachers valued the ERWC curriculum, finding its activities and 
readings to be effective and useful, especially in promoting student engagement. 

ERWC teachers reported that the course elicited high levels of student interest, making it 
easier for them to keep students on task and get students to participate in class activities. 
Representative teacher comments follow: 

[Module 8, Juvenile Justice,] is another extremely engaging topic that the vast majority 
of students can buy into. It makes the engagement piece much easier for me as a 
teacher, and just reminds me that the best way to improve student ability is to get them 
to interact with the material and be interested in it. 

[Module 5, Good Food/Bad Food,] is a very relevant and engaging module for the 
students. They bought into the topic and the assignment and I had little difficulty in 
keeping them interested and engaged throughout the process. 

Many teachers reported that student engagement led to “great,” “fantastic,” or “thought- 
provoking” classroom discussions. One teacher reported that students “engaged in lively 
conversation among their groups, and I had time to sit in on their conversations. Then, we 
were able to conduct a reasonably good whole class discussion which involved students 
defending their choices of intellectuals.” 

Another teacher described the high level of class discussion with regard to module 2, The 
Rhetoric of the Op-Ed Page, noting that one particular class discussion had led him to 
reevaluate the manner in which he plans to facilitate future student discussions: 

There was a lot of buy-in to the topic of this module, much more than I anticipated. At 
the start of the module I had a class discussion like I usually do, but I had so much 
engagement and desire for students to participate that I think next time I will have a 
class debate instead of just a conversation. 

Class discussions further contributed to student thinking about thought-provoking topics, 
challenging their perspectives on certain issues. One teacher noted that, as a result of 
engaging in discussions and listening to the perspectives of their peers, some students 
changed their stances on an issue. More specifically, the teacher noted that the topic of 
juvenile justice had evoked strong emotions among students, with some initially 
expressing anger at the main character, but that students’ perspectives changed over the 
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course of the module, suggesting that the material, coupled with classroom conversations, 
had stretched their thinking: 

[Students] were quite engaged in their discussions, often changing their perspectives as 
they went and as more information was presented. Most were quite angry at Greg Ousley 
at the beginning and then began understanding why he did what he did and how he had 
changed since then. ... In the same way, they all at first said that all teens should be tried 
as adults and held in prison their whole lives. And, again, as they saw facts ... their ideas 
changed. It is always fun, as an educator, to challenge students' thinking and to see 
them changing their beliefs based on strong evidence. 

One teacher even said that she allowed students to challenge her own position, something 
the students “really enjoyed ... and gained so much from.” 

Teachers indicated that, at least in part, module topics were interesting to students 
because they prompted students to think about their own futures. For instance, module 1, 
What’s Next, provided students with an opportunity to learn more about college options 
and career choices. Another teacher noted that module 4, The Value of Life, prompted 
students to think more deeply about their future overall: 

I felt that this module captured the attention of the students the most thus far. The 
articles about Ebert and Jobs were engaging and made the students really dig into their 
own thoughts about their future. 

Teachers also expressed appreciation for the course readings, often referring to the 
literature as “excellent,” “great,” or “useful.” Several teachers specifically mentioned liking 
Into the Wild from module 6 of the same name. One teacher described the book as well 
suited for expository purposes, and another noted her students’ high level of engagement 
with the main character, Chris McCandless. Another teacher speculated that, as high 
school seniors, her students could relate to the main character: “This was my first read of 
Into the Wild, and I found it to be a very enlightening story, especially for students at this 
time in their life. Multiple parallels exist between them and McCandless.” 


Finding 2: Teachers learned new content and strategies that changed 
their practice. 

In the course of teaching the ERWC, teachers reported that they learned new teaching 
strategies, the most frequently cited being those aimed at improving student writing. For 
example, one teacher stated, “I enjoyed learning the new strategies — Cubing 40 and PAPA 


40 Cubing is an instructional strategy that asks students to consider a concept or vocabulary word in a 
variety of different ways. When students consider the concept in different ways, each side of the cube 
has a different prompt (e.g., describe it, compare it, associate it). 
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Square 41 ... the PAPA Square provided a strong basis for writing the precis.” Other teachers 
mentioned having learned strategies to help students write rhetorically. For example, one 
teacher noted that he learned “how to focus on the rhetorical strategies as a tool to 
improve writing.” Teachers noted learning other new skills and concepts as well: 

I am also learning a great deal about teaching grammar — the text-based approach is so 
effective and the focus of the lessons is particularly well suited to the needs of 12th grade 
writers. The lessons facilitate the revision process effectively. 

I learned how to emphasize skills while covering concepts in teaching a novel. This is 
quite a shift for me because in the beginning of my teaching career, everything was all 
about the novel, but now I'm really understanding how to focus on discrete 
reading /thinking /writing skills. 

Teachers reported that the ERWCs underlying strategy of giving students greater choice 
and independence in their learning experiences resulted in higher student engagement. 
When students were given the opportunity to explore a topic of interest to them, they 
were more invested in their learning, as teachers reported: 

I learned that there is inherent value in allowing students to explore their own topics. 
There's definitely more of a student buy-in, especially when it comes to writing 
something as extensive as a research paper. 

[I learned] to allow students to build their own context and do their own research. It is 
far more meaningful if they ’’self-discover. ” 

Having students bring in texts they choose helps create interest and makes them feel 
invested in their education. 

This teacher feedback highlights the importance of allowing students to conduct research 
and find their own answers. Teachers reported that, in addition to fostering student 
learning and growth, this approach gives teachers a greater understanding of students’ 
interests. In fact, they reported that teaching the ERWC modules enabled them to get to 
know their students on a deeper level, to develop a greater understanding of students’ 
academic and career goals. Thinking of what he had learned about his students by 
teaching module 1, one teacher wrote: 

I learned that many of my students are not as prepared for life after high school as they 
thought they might have been. They had never considered looking at their futures the 


41 PAPA Square, whereby students analyze the purpose, argument, persona, and audience of a text, is 
both a reading and a writing strategy. Students answer a series of questions in response to the text that 
they are reading or their own writing, such as: What is the purpose? Who is the audience? What persona 
is the author, or am I, trying to project? What is the thesis or argument? In the center of the PAPA 
Square, students identify the stylistic devices and the emotional, logical, and ethical appeals used to 
persuade their audiences. These may include types of evidence, figurative language, text structure, or 
tone. 
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way they had to in order to complete the module. I also learned a lot about my students 
... their fears, strengths, personal challenges. It helped me connect with them on a much 
more personal level. 

Another teacher echoed this sentiment, describing the importance of the “What’s Next?” 
topic in module r in helping students to voice not just their plans and aspirations for the 
future, but also their apprehensions. The teacher described how the ERWC provides an 
opportunity for teachers to reassure students about their futures by building a sense of 
community in the classroom: 

I learned how very fragile and uncertain many of my students are at this very critical 
time in their lives. The reflection process is not something that they often have time to 
do, and this module provided them with an opportunity to express some of their 
apprehensions. Seniors require a lot of reassuring and, for our population, school is more 
times than not the place they receive [that reassurance]. They have questions about the 
next steps in their lives, and if a prior relationship has not been formed, it is difficult to 
ask a teacher you just met for assistance in these areas. The module was a great 
community -building exercise. 


Finding 3: Teachers reported that many students began the school year 
unprepared for the reading and writing demands of the curriculum. 

Students who were enrolled in the ERWC during the study represented a wide range of 
reading levels and English-language abilities, but a common concern reported by their 
teachers was that many students were unprepared for the rigor and writing-intensive 
nature of the course . 42 A common theme in teacher feedback was that students had not 
been sufficiently exposed to the types of skills required for success in the ERWC. One 
teacher noted that many of his students began the course with absolutely no knowledge of 
rhetoric or its features, and suggested that this might change if ERWC strategies were 
“implemented in earlier grade levels.” Another teacher echoed this notion: 

I learned that, as a school, we need to work on [having students write] essays of all 
modes from the get-go. We need to work more intently ... to instruct them on how to 
select meaningful passages as evidence and then how to explain the significance of these 
passages as support of their theses. But, this process needs to begin earlier than 
12th grade. 


42 As stated in the University of California English Course Proposal for the California State University 
Expository Reading and Writing Course (2013/14), “The goal of the Expository Reading and Writing 
Course (ERWC) is to prepare college-bound seniors for the literacy demands of higher education.” 
However, enrollment in the ERWC is not restricted to college-bound seniors. This has implications for 
the scaffolding of the curriculum and the pace at which a teacher will cover the modules. 
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Reading comprehension was one area in which many students reportedly struggled. One 
teacher noted that many of her grade-r2 students read at the grade-8 level, making it 
difficult for them to engage in the course reading: 

... Almost everything [has to be] be read together since [students’] instructional reading 
levels are so low. It takes an enormous amount of time, effort, and energy to do this in 
class with them. I would love to have a class of students who could read this at home and 
come prepared for discussion, but that is not the case. For this reason, we had to forgo 
certain activities, especially the rhetorical grammar, which I know they need. 

Reading was often complicated by vocabulary that the students found challenging. Several 
teachers reported spending more time than anticipated guiding students through word 
definitions. Referring to activity 3 in module r, one teacher described how she asked 
students to highlight words that they did not recognize: “This activity took much longer 
than expected as students were not at all familiar with several terms. We had to define 
most words before continuing with the activity.” 

Student writing was another common teacher concern. Writing-related concerns ranged 
from students’ grammatical errors to the structure of their essays to their inability to 
effectively incorporate research into essays. Some students also struggled with the amount 
of writing expected in the course, as well as with writing under pressure, as in the course’s 
timed writing assignments. Additionally, some teachers reported that students did not 
always understand the writing prompts, identifying that as a significant obstacle in 
students’ ability to meet expectations on writing assignments. Students’ written work also 
suggested that, prior to enrolling in the ERWC, many students had not had sufficient 
exposure to the various writing styles or genres presented in the ERWC curriculum. 

Teachers observed that, generally speaking, their incoming students needed a lot of 
improvement in their writing skills. One noted that his students lacked the basic 
grammatical skills to accomplish the tasks in module r. Similarly, another teacher noted: 

... Grammar and mechanics were very weak. Students are still writing run-on and 
fragment sentences, creating comma splices, and using cliches and poor diction/weak 
verbs, even though we have completed grammar activities on this. These errors were 
across the board in three classes. Now I know the areas that need more work. 

Teachers also noted that many incoming students struggled to write basic paragraphs, as 
well as to summarize and paraphrase. Referring to the second rhetorical grammar activity 
in module r, one teacher said he became aware that many students struggled to rewrite a 
paragraph: “Many ... just focused on paraphrasing and then trying to make their notes 
match the original, but many just got stuck and weren't able to write a coherent paragraph 
again.” 
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Teachers also reported that students needed more guidance on the format or structure of 
an essay. One teacher noted that, after reading students’ writing samples from module 2, 
she realized that she needed to review basic essay instructions with her students: 

I have learned that many of my kids need help just in simple essay writing. MANY of 
them do not know how to write a thesis statement; many of them don't know how to 
support their argument with reasons and examples; many of them think that three 
sentences are enough for a supporting paragraph. I am going to have to go back (at 
some point) and teach just simple essay writing strategies. 

Teachers also found that incoming ERWC students had not had enough prior exposure to 
different styles of writing, such as expository and rhetorical. Most importantly, many had 
not had much, if any, experience writing a research paper. Several teachers reported that 
students did not provide enough evidence in their research papers. Referring to module 8 , 
one teacher wrote that she is “still finding students who are unable to support their claims 
with relevant textual evidence, or that don't use any at all.” 

Teachers also noted students’ struggles with rhetorical writing. For example, one teacher’s 
feedback for module 2 indicated that his students were unfamiliar with “op-ed” pieces, 
had “relatively little experience in rhetorical writing strategies,” and did not understand 
how words could be used to influence people. 

Finally, some teachers reported that students were intimidated both by the amount of 
writing and by the need to write under the pressure of time. Referring to the essays that 
her students had submitted after a timed assignment, one teacher wrote: 

OMG, I wanted to scream! After all the time we’ve spent on addressing prompts and 
writing under constraints, when given the prompt for an in-class essay, 80% of students 
panicked and reverted to old habits. Their essays were off-topic. They did not properly 
address all aspects of the prompt ... 

Finding 4: Over the course of the school year, many, though not all, teachers 
reported observing student growth in the skills introduced through the ERWC. 

While students’ struggles, particularly early in the school year, were noted by teachers, the 
teachers also reported seeing improvements in reading and writing over the course of the 
year. With respect to reading skills, teachers noted the increased levels of careful reading 
that students demonstrated. One teacher observed, for example, that, in module 10, 1984, 
“students were able to access the content in terms of plot and making connections. They 
were also able to employ different reading strategies to support and enhance that 
understanding.” 

Additionally, teachers saw growth in students’ analytic skills, as one teacher noted that 
her students demonstrated analytic skills when doing a “chunking” exercise for module 6 : 
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[My students] did a very good job understanding the rhetorical strategies used by the 
author, as well as looking at the structure of a piece of writing. This was evidenced by the 
students' analysis of the various chapters, as we chunked each section into specific parts 
of Chris' life. 

Teachers also commonly discussed improvements in writing skills over the course of the 
year, noting that the majority of the skills that students employed in their writing 
stemmed from the ERWC activities. For example, teachers commented on students’ 
increased ability to use references. One teacher commented that “students did well with 
their MLA parenthetical citations and works cited page. They also structured and 
organized the content of their assignment. Students drew in information from their 
sources.” 

Students’ improved writing skills were on display in final essays, notebooks, and outlines, 
among other writing vehicles, as noted by several teachers: 

Based on their final drafts of their personal essays and a few of the letters of 
introduction, my students met the objectives of managing information, applying the 
rhetorical framework to reading and writing, understanding writing as a response to an 
audience /situation /intention, and writing clear and coherent prose. 

For the most part, they did well; they wrote convincingly with understanding and 
opinions of their own. The evidence for this was their extensive writings in their journals 
and in response to the many questions in the various activities, and of course, their final 
exam essays ... 

My students met the objectives of reading informational texts, analyzing the use of 
textual features, citing strong textual evidence to support analysis through their 
notebook questions/ answers, their final short answer responses, and their class 
discussions/Socratic Seminars. 

Students’ improved analytic skills, as well as better articulation of their ideas, carried over 
into presentations and discussions. Teachers commented that students engaged in 
insightful discussions and created well-thought-out speeches, letters, and public-service 
announcements. One activity in which these skills were manifested was discussion in the 
Socratic Seminars. In reference to module 8 , one teacher stated that “after so many 
modules of practice, students were able to find their voice and contribute meaningfully 
during our Socratic Seminar. I thought giving students an opportunity to demonstrate the 
skills they have learned ... was beneficial and helped them to internalize those skills.” 
Another teacher commented that “students understood the authors' tone and content well 
enough to imitate their style. I thought they did a good job on this activity.” 

Overall, teachers thought their students gained a wide variety of skills through the ERWC. 
Referring to module 12, Bullying, one teacher wrote: 

I think my students went above and beyond the objectives of this module. They not only 
read and annotated the assigned readings of the module, but they read and annotated 
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readings for their own topic as well. They conducted interviews and watched videos, 
tested the reliability of websites, and wrote amazing papers. They asked and searched for 
an answer to their own research question. 

In addition, teachers spoke of students demonstrating improved critical-thinking skills: 

I was very pleased; I felt that the students displayed stronger critical-thinking skills than 
I had seen in the fall. They did a good job of referencing the different articles and 
connecting to the points they were trying to make. 

Students exhibited growth in other ways as well. One teacher observed that “it was nice to 
see that we had built a strong enough foundation that the students were beginning to take 
over some of the responsibility for what they were learning.” Ultimately, this meant that, 
toward the latter part of the school year, teachers did not have to spend as much time 
reviewing the strategies or prompting students to make use of them. One teacher’s 
enthusiasm comes through in her comment about student progress: 

[In] looking at my students' work from this module I learned that many of my students 
are taking the things that I have been teaching them and focusing on and are USING 
them in their timed writing! It is nice to see. Now, not ALL my kids are progressing at the 
same levels, but I definitely see growth across the board. 

Various teachers attributed the growth in students’ skills to the ERWC activities. In their 
view, students gained deeper understanding and started to make more connections 
between topics as a result of the concept map, activities on contextual clues, and 
analyzing texts from different angles through looking at stylistic choices and thinking 
critically: 

Based on my students’ work, discussions, formative assessments, and end-of-the- 
module writing, my students have definitely improved. They may not be 100% proficient 
in everything, but compared to the work that they were able to produce at the beginning 
of the year, they have made great strides. 

Students are becoming strong writers and are using many of the writing techniques we 
have learned on their rough drafts as opposed to later on in the edit process. 

Their essays were some of the best pieces of writing of the year [module 11, Brave New 
World^. Somehow their level of seriousness when writing and editing was very strong. A 
number of students struggled to understand the writing task, but even they did their very 
best with careful editing and revising. 

One teacher described her students’ improved abilities to write lengthier papers in 
module 6 and attributed this improvement to holding students accountable to high 
expectations: 

At first my kids were SHOCKED that they would have to write a 5 PAGE PAPER ... but in 
the end, many of them had a hard time keeping it to 10 pages. ©I definitely re-learned 
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that if I, personally, put the rigor into my instruction and I hold my kids accountable to 
the highest standards, most of them will rise to the occasion. 

This challenging opportunity for the students to write a lengthier paper resulted in a “re- 
learning” experience for the teacher, about students’ potential to meet high expectations. 
Another teacher stated that the essays submitted by students for module 8 were “the best 
examples of writing I received from them all year.” In her feedback on module 7, Bring a 
Text You Like to Class, another teacher spoke of her students’ growth in reading and 
writing: 

Based on their writing and reading skills demonstrated during this module ... my 
students are making [fewer] errors and are able to handle more complex texts on their 
own. We did have to spend more time during the writing process than the module called 
for, but their portfolios turned out quite good and they were very proud of their work. 
100% of my students completed their writing project. ... I was able to see the most growth 
in my students f] reading and writing. They are now able to cite, define and implement 
strategies, utilize academic language when appropriate, and not only know the writing 
process, but can independently follow it. Yes, they still have areas of [needed] growth, 
but their improvement has been considerable. 

Some teachers who commented on ways in which students struggled at the beginning of 
the year noted student growth in those same areas toward the end of the year. For 
instance, one teacher initially commented that students weren’t able to write a coherent 
paragraph after paraphrasing, and later said, “I feel like the majority of my students have 
been steadily improving in the quality of their writing since the beginning of the year.” 
Another teacher noted that her students’ instructional reading levels had been very low at 
the beginning of the year, and commented on module 9, Language, Gender, and Culture, 
that “students all wrote proficient precis, and they could find a claim and map out the 
structure of the articles.” Another teacher, who had commented about students’ difficulty 
accessing a text at the beginning of the year, described how her students had grown: 

What I was looking for here was transference. Were students understanding how 
strategies are applicable cross-curricularly ? Some still require you to focus them on 
applying what they already know, while others ask for permission to employ useful 
strategies to new situations. It is refreshing to see how much they have grown. 

In general, teachers noted seeing student growth and progress, with one teacher 
mentioning: 

All of my students have definitely made academic progress. They are better students, 
have better habits of mind, and have more writing and reading tools and skills than when 
they started this course at the beginning of the year. 
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Finding 5: Many teachers reported challenges with the pacing of the course. 

The most frequently noted challenge in implementing the ERWC was the amount of 
material in the curriculum. 43 In essence, a major concern for the teachers related to 
pacing. One teacher described the stress that she experienced in attempting to cover all of 
the activities in module 1: 

Wow ... I definitely learned that I should NOT try to teach the modules in their entirety. I 
honestly tried to complete each activity as thoroughly as I could and to fit in all the 
activities; in doing so, I feel like I burned myself out ... I skipped a lot of the last [half of 
the module and only hit the "biggies." 

Some teachers shared that their students were also concerned about the number of 
activities. Referring to module 2, one teacher wrote: 

My students overwhelmingly complained about the number of activities contained 
within the Rifdn piece. I agree. 

Another teacher shared her opinion about the number of modules that teachers were 
expected to complete during the school year: 

I think overall that 8 modules [are] too many for one school year. There are always 
things that come up that cause a teacher to lose days. I think it should be 6 modules a 
year, and this would allow the teacher to go more in-depth. I asked the students to write 
an evaluation of the course, and that was a big thing most of them wrote about; not 
going enough in-depth. 

Other teacher concerns related to perceived redundancy, with several teachers describing 
activities as repetitive within some modules. One teacher noted that, while module 1 had 
some “great activities ... some are a little repetitive and others need modification to not 
disturb the flow of the classroom.” Another teacher described the questions for activity 14 
in module 1 as unnecessary and difficult to address. She wrote, “Already this feels like it is 
getting repetitive and some of the questions I found slightly unnecessary or really hard to 
answer just by looking at the text.” These comments suggest that teacher preparation for 
the course may need to more explicitly state that teachers have the flexibility to pick and 
choose which activities to teach in order to avoid repetition. 


43 The implementation feedback charts gave teachers the option of reporting that they did not teach a 
particular activity, and the charts provided a number of options for teachers to indicate why they did not 
teach the activity, including “Students had already mastered this knowledge/ skill,” “I combined this 
activity with another activity in this module,” and “Our class did not have enough time to work on this 
activity.” A version of the hard-copy implementation feedback chart for module 1 is provided in 
appendix B. 
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Finding 6: Some teachers reported that students did not always understand the 
writing prompts. 

One teacher stated that her students were not successful in their essays for module 4, The 
Value of Life, because they did not understand the “open nature of the writing prompt.” 
She said that students wanted more specificity in the prompt. Another teacher echoed this 
sentiment, noting that until her students submitted their essays for module 8 she had not 
realized that they did not understood the prompt, “which makes me think that they were 
more concerned with the stories than the issues raised by the stories.” She added, “I need 
to focus more on the issues and focus more on the writing prompt.” A third teacher also 
expressed the need to work more closely with students on understanding the writing 
prompt and providing them with opportunities to edit their work: 

One big issue that I had ... was the students not understanding the prompt they chose. 
Instead of responding to the Steve Jobs article and using the other readings as their 
evidence for their claims, they just gave me a summary of what Steve Jobs said and used 
textual evidence from his own address. Their essays were still organized well, they just 
did not respond to the prompt that they chose. If I had more time with them to do a 
process paper, this is a mistake that would have been pointed out and [that] they would 
have fixed in their subsequent drafts, but unfortunately I just did not have the time. 

Finding 7: Teachers reported that they encountered technology impediments when 
implementing the ERWC. 

Some activities within the modules required the use of a computer; for instance, an 
activity may require students to use the internet to search for scholarly articles. Some 
teachers noted running into technology impediments that precluded them from 
completing these types of activities. For instance, some teachers indicated that their 
school’s computer lab was unavailable, in some cases for long periods of time, due to use 
by other classes or events taking place at the school. One teacher reported that her classes 
were not able to go to the computer lab because “at the beginning of the year, the 
computer lab is used for freshman orientation, [which] takes quite a bit of time.” In other 
cases, teachers’ classes had access to the computer lab, but the Internet was not working 
on the day, or days, that students were due to be in the lab. For example, one teacher 
stated, “When I had the library and computer lab scheduled to work on the writing 
assignment, the Internet was down ... so my librarian was able to pull an entire cart of 
books on controversial topics.” Another teacher reported that a districtwide Internet 
shutdown precluded her students from using computers. 

Some teachers reported skipping activities as a result of technology limitations. For 
example, one teacher stated that his class did not have an opportunity to use the 
computers in the school library, and because some students did not have Internet access 
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at home, he opted not to assign some of the activities in module 7, Bring a Text You Like 
to Class: 

I didn’t have the opportunity to go to the library and didn’t feel comfortable assigning 
these activities for homework when not every student has easy access to the Internet or a 
library to find scholarly articles. So I did not do the rest of the activities. I did consider 
finding a few scholarly articles to share with the class, but I felt that defeated the whole 
purpose of finding a text that related to a topic. 

Finding 8: Most teachers reported that they made some modifications to the ERWC 
curriculum. 

Teachers commonly reported modifying the curriculum by combining activities within a 
given module or by changing the order of activities. For example, one teacher stated that 
in teaching module 5, she “changed the order of certain activities and combined others to 
make it make sense.” Another teacher stated that she and her colleagues “feel that having 
the rhetorical grammar at the very beginning interrupts the kickoff for the module. We 
prefer to combine the activities later in the module when student interest needs a change 
of pace.” In other words this teacher moved the rhetorical grammar activities from the 
beginning of the module to later in the module, and she combined some of these 
rhetorical grammar activities together. 

Many teachers modified the ERWC curriculum by assigning activities as homework. 

Others wrote about making changes to writing assignments, such as modifying the writing 
prompts or changing the amount of time for completing a writing assignment. For 
example, one teacher described that he changed the writing prompt for module 1 in 
response to his students’ requests that the prompt align with the prompts of the 
universities to which they were applying. Another teacher indicated that she elected to do 
an in-class essay instead of fulfilling the writing expectations at the end of the module, 
because the expectations felt “daunting.” One teacher modified module 8 by changing the 
writing assignment into a mock trial: 

While I found the idea of the essay very useful, I decided that I could still implement the 
reading, writing, and speaking elements from the essay assignment and transfer those 
skills into our ‘Mock Trial. ’ 

The reasons behind the modifications varied. For instance, some teachers focused on only 
certain readings, in an attempt to cover the eight modules required for the course: 

The materials were extremely accessible for students, but the module in its entirety is too 
long. The decision to focus on only one text was made at the advice of our ... coach, in an 
effort to gain time needed to complete the required four semester modules. 
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Other teachers modified the curriculum to meet the needs of students in their classes. As 
one teacher reported, “I know I’ll always modify activities based on personality and ability 
level of each individual class.” 


Finding 9: The majority of teachers supplemented the ERWC with materials of 
their own. 

Teachers commonly supplemented the curriculum with videos from such sources as the 
Public Broadcasting Service (PBS), YouTube, and television shows. Teachers indicated that 
they sought out videos to provide students historical context on the curriculum. For 
example, a few teachers reported that they used video on Roger Ebert to introduce 
students to the television personality and his role in media. One teacher explained her 
rationale: 

I ... went to YouTube in order to show students excerpts from “At the Movies” with Siskel 
and Ebert. I had to provide a lot of background for Ebert because most of my students 
had no idea who he was. 

Another teacher used footage of September 11 for module 4 so that students would have 
additional context for the significance of the event about which they were going to read: 

The majority of my students had much to share on this topic ofg/u and their 
response/connection to this event. Most of their comments focused on those people 
(usually adults) around them and how these older people reacted rather than how the 
students themselves (who were very young at the time) processed the terrorist attacks 
themselves. As a result, I showed the class a short u-minute video clip that recalls that 
day and the impact on our nation in order to give the students a better understanding of 
what we were about to read. 

Teachers commonly used their own instructional tools to supplement their teaching of the 
curriculum. Several mentioned using graphic organizers that they created to assist 
students in completing the activities more effectively and in developing essays, and several 
indicated that they employed additional tools to help students access challenging 
vocabulary: 

As in the other modules, I felt my students needed more support with vocabulary, so I 
created supplemental materials and activities to help us get through a text full of 
challenging vocabulary. 

Students in our ERWC classes are required to keep a personal dictionary of words and 
phrases that they don't know. This activity was used as a quiz on the effectiveness of 
their personal dictionaries. Students were allowed to use their personal dictionaries to 
complete the assignment. Words that were not included in the dictionary should have 
been familiar to them. Others, they would have already added to their dictionary and 
annotated within the article. 
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ERWC Coaching 

The i3 grant funded teacher coaching by ERWC experts, as part of the course’s ongoing 
professional-learning activities. Coaches provided a range of support to ERWC teachers, 
from constructive feedback, related to overall teaching practices, to teaching specific 
concepts or skills, to leading and facilitating holistic scoring sessions during PLC 
meetings. As coaches identified and worked with teachers to address areas in need of 
improvement, highlighting the positive, in terms of teacher practices and student 
engagement, was central to the coaching model, both to foster strong relationships and to 
reinforce best practices. 

Each ERWC teacher was expected to meet with a coach twice each semester (four times 
each year). The majority of these coaching sessions consisted of the coach observing the 
ERWC teacher, with time for discussion after the observation . 44 After each visit, coaches 
completed a coaching log, which was typically one page in length (the coaching log 
template is provided in appendix B). The majority of these logs captured coaches’ 
observations of classroom practice, along with summaries of their conversations with 
teachers. The coaching logs also provide important insights regarding the coaches’ 
perspectives on the implementation of ERWC by the teachers. This section summarizes 
major themes emerging from the qualitative information about the coaching component. 

Finding 10: Overall, teachers had positive experiences with the ERWC coaches. 

As captured both in the coaching logs and in end-of-year ERWC teacher surveys, the 
coaching provided a range of support for teachers, including feedback on their teaching; 
advice and support on implementing specific teaching strategies; guidance and expertise 
related to the ERWC curriculum; and opportunities to share openly and ask questions. As 
one teacher noted on the end-of-year teacher survey: 

I benefited greatly from my coach’s expertise on many levels: 1) Useful constructive 
advice on my teaching strategies he observed during his classroom observations; 2) He 
was quick to send me any information about any topic I requested; 3) He knows the ins 
and outs of the curriculum and could answer any question with great insight. 

Coaches noted their efforts to develop a trusting coaching relationship with teachers. 
Some of the strategies that they used to achieve this goal were focusing on one or two 
suggestions per session; finding behaviors or experiences to recognize positively in each 
meeting; and pointing out where students were successful, to highlight what students are 
capable of doing. 


44 Coaches would also occasionally attend PLC meetings to provide additional support to teachers. 
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In their end-of-year survey responses, the ERWC teachers emphasized how constructive 
feedback and advice helped them to reflect on their own teaching practices and beliefs 
and strengthened both their skills and their confidence in teaching. As one teacher 
commented, with respect to the coaching, “I gained much insight into my own teaching, 
especially with regards to formative assessment, engagement, and motivation.” Similarly, 
another teacher wrote, “It’s nice to have an extra set of eyes in the classroom and to 
receive tips and feedback from another professional who is knowledgeable about [the] 
ERWC as well as about teaching writing.” A third teacher said, “It was helpful to have an 
observer to point out strengths and weaknesses in a non-threatening way. It was also 
helpful having a resource for difficult tasks and/or text. She helped with the ‘why?’” Lastly, 
a fourth teacher commented, “Being able to get an unbiased opinion about what I’m doing 
well and what I need to improve helped me tremendously.” 


Finding n: Coaches provided a range of support to teachers. 

Coaches often focused their coaching to promote ERWC strategies. As one coach noted, 
with respect to using a rhetorical approach to reading: 

[The teacher] and I had a conversation /, ] after her lesson [, ] about her literary approach 
to the reading. This reflects her own preferences and enthusiasm, and she is somewhat 
resistant to the rhetorical approach called for in the modules. We discussed this, and she 
has a better understanding of that approach and will try to more thoroughly embrace it 
in future work. 

Coaches also provided guidance on activities: 

It seems to me that asking students to do a rhetorical precis was not the best solution to 
the issue of their inadequate analysis. I will redirect the teacher to the module and the 
activities designed to engage students in critically analyzing the articles [from 
module g]. 

In another instance, a coach guided a teacher to change an assignment that she had been 
planning and that was not in the module she was teaching (module 6, Into the Wild): 

[The teacher] planned to have students use the book to find examples of the existentialist 
influence in Chris’s life [from Into the Wild/. We discussed the importance of students 
being engaged in the reading and of using the ERWC modules as designed. We talked 
about what would be appropriate changes to make and what might get too far away 
from the design of the course. 

Coaches also supported teachers on other aspects of the curriculum, such as 
understanding writing prompts: 

Teachers had difficulty deciphering the essay prompt and did not have a full 
understanding of the articles ... I walked them through the goals of the module and asked 
them to invite me to the PLC meetings when they were discussing any future modules 
about which they might have questions. 
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One coach described working with a teacher to employ reading quizzes that were more 
accessible to her students and that would hold them accountable for completing the 
reading assignments: 

We talked about simplifying the reading quizzes. She agreed that the one she gave to 
them was too difficult. I suggested she base a reading quiz on the reading-for- 
understanding activities that were built into the Brave New World module. These could 
be quick quizzes that would hold students responsible for their reading. 

Other teaching practices addressed by coaches were classroom management and time 
management. More specifically, coaches worked with teachers on effective ways to 
respond to student behavior issues: 

[The teacher] asked me if I had any specific advice on how to go about focusing the 
students, and I reminded her that activity 26 included model questioning, where the 
teacher models how to ask questions of an introduction and then students pair up and 
ask questions of each other’s questions. She decided immediately to lead with that and 
began class with the modeling of questions for “When I was in eighth grade I did not 
know how to read. ” 

[The teacher] has already implemented randomly calling on students as well as giving 
students points for participating and taking points away for not participating. There 
were three students in the back who seemed to talk the whole time, and we discussed the 
possibility of breaking them up or putting them closer to him. We also talked about 
trying to make these students his allies, as others in the class might follow their lead if 
they are good in class. 


Finding 12: Coaches generally reported observing high student engagement in 
the classrooms. 

While coaches reported seeing variation in students’ levels of engagement during 
observed sessions, they noted that, more often than not, the majority of students appeared 
engaged with the lessons. They attributed this engagement to a range of factors, foremost 
that teachers planned lessons that addressed areas of student need. Coaches noted that 
the topics and types of activities covered in a lesson, as well as the teacher’s teaching style, 
also impacted students’ engagement. They found that when teachers tailored lessons to 
address students’ needs with respect to, for example, practicing timed writing or 
understanding the text, students seemed more engaged: 

The lesson was very successful; it was itself an attempt to address difficulty students 
were having with timed writing ... Almost all of the students seemed engaged because 
[the teacher] was answering a need that they had expressed. 

To address the vocabulary challenges, [the teacher] continues to work on vocabulary 
development by defining unfamiliar terms that students encounter and interacting with 
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students over unfamiliar words ... He also found the PAPA Square strategy helpful with 
analysis and vocabulary. 

However, a few coaches noted that tailoring lessons to meet the needs of specific students 
in a given classroom can cause teachers to fall behind the schedule for completing 
modules. Coaches worked with teachers to help find the right balance of scaffolding and 
pacing activities within a given module, to ensure that students’ needs were being met and 
that the class as a whole was making adequate progress. 

Student engagement was sparked not only by lessons that met students’ needs, but also by 
the topics covered over the course of a module — a point also noted in the previous 
section of this chapter, describing teachers’ observations. One coach noted, “The subject 
of animal versus human rights provoked a strong emotional reaction in many students; 
some were more vocal than others.” When students were able to connect with the topic — 
either because they saw the personal value or because they were having an emotional 
reaction to it — it motivated them to participate in the various classroom activities. In one 
classroom observation, a coach noted, students seemed drawn to the story in the book 
Brave New World (in module 11), and this increased their engagement: “Students started to 
quiet down more when they started reading the text in class. Students were interested in 
the story.” In another class, a different coach documented students’ interest in the topic of 
Language, Gender, and Culture in module 9: 

Students are very interested in the topic, so they will look things up. This means, they are 
entering a conversation with more research; they are annotating text and they are 
improving. 

In addition, the types of classroom activities also influenced student engagement. Coaches 
noted when teachers strategically structured an activity to promote student engagement, 
whether that structure involved time for individual reflection and writing, sharing in pairs, 
or discussing an issue as a small or large group: 

Students were highly engaged. [The tjeacher varied sharing techniques from face-to-face 
partner to elbow partner to random calling on students. 

Students appeared to be quite responsive to [the teacher’s] request for participation. She 
has various methods of engaging students, besides theA/B pairings that she’s arranged. 
She asks good questions about texts, and she gives students in-class time for 
Quiclcwrites. She then called on students to share their writings, and they were 
interested in doing that. 

The most commonly noted reason for students to be disengaged was repetitiveness of 
certain activities. One coach commented, “Students are finding the sharing somewhat 
repetitive ... [The teacher and f] discussed the need to help students focus on strategies as 
a way to deal with concerns about repetitiveness.” Coaches regularly helped teachers to 
think through and plan for lessons that would increase student engagement by 
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responding to student need, covering topics that are personally relevant or interesting to 
students, and including a variety of activities. 

Finding 13: Coaches noted that some teachers encountered challenges related to 
assessing student work. 

Coaches noted some discrepancies in the ways student work was assessed across teachers. 
For instance, a coach discovered that two ERWC teachers at a particular school had very 
different expectations with regard to scoring student writing. The coach noted that one 
teacher told him that when she had given one student’s paper a score of o, the other 
teacher at the same school had said she would have given the paper a score of 70 because 
the student was an English learner and “it’s hard for her.” The coach concluded that “the 
two teachers may have vastly different expectations with regard to scoring student 
writing.” 

That same coach became aware of yet another layer of challenges when, in reviewing 
student papers graded by the first teacher, the coach found that most of the teacher’s 
corrections were for grammatical problems, mainly addressing what the coach called 
“surface-level errors,” such as “no contractions” and “no i st person pronouns.” The coach 
also noticed that the problems in the papers that this teacher had graded as “zeros” 
seemed to be due largely to a language issue, rather than to the students’ ability to 
conceptualize the topic. The coach attempted to identify these concerns to the teacher 
and suggested that students receiving such poor grades be given the opportunity to revise 
their papers: 

When I point this out to [the teacher], she says that it doesn’t matter — if it doesn’t 
come across in the first read, then it isn’t therefor the reader, and she doesn’t have 
enough time to reread and figure it out. This goes to the problem of teaching a writing- 
intensive class like ERWC with an overload of students. With the large numbers of 
papers to grade and a goal of preparing college-ready writers, [this teacher] is opting for 
writing as a summative evaluation rather than as an opportunity to engage with the 
writing and help [a student] develop further. 

This finding also highlights the coach’s view that it is important to give students an 
opportunity to improve their writing skills by revising their papers in response to teacher 
feedback, and indicates the coach’s concern that the teacher is more preoccupied with 
summative assessment and less so with utilizing formative assessment to identify and 
work on areas where students may struggle. At the end of the coaching log, the coach 
states that, upon the student’s request, the teacher was willing to allow the student to 
revise the paper. 

To try to address discrepancies in grading across teachers, some coaches conducted 
holistic scoring sessions with teachers, to calibrate the grading on student essays. In a 
series of coaching logs, one coach described her work with two teachers whose scorings of 
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student essays were at odds with each other. The coach initiated a holistic scoring session 
with the two teachers and selected what she referred to as a set of anchor papers. The 
coach then removed student’s names from the papers and made copies for both teachers. 
The coach described the experience in the coaching logs: 

After we had finished with the anchor papers, we scored some individual papers. [One 
teacher] and I scored some of the same ones so that she could continue to calibrate her 
scoring to the rubric. We noticed that she tended to score the papers higher than I did. I 
[suggested] ways to respond to the papers, because she noted that if she didn’t mark 
anything, students would ask her why they had gotten the grade. I suggested that she 
highlight the squares in the rubric and then write a comment of specific praise and a 
suggestion for next time, or for a rewrite if one was warranted. 

The coach went on to say that she believes these teachers may change their grading 
practices moving forward, “ [i] f only to mark slightly fewer errors and to calibrate more 
carefully to the rubric instead of being swayed by effort or grammar.” 

Finding 14: Coaches identified some challenges associated with fidelity of 
implementation, as well as with pedagogy. 

Some coaches indicated that some teachers taught the ERWC in a way that seemed 
disconnected from the ERWC framework — that the teacher’s approach did not make 
explicit the connections between the modules or the connections between activities 
within a given module. For example, coaches reported that some teachers taught the 
ERWC as though it was just a series of activities: a teacher would go through the sequence 
of activities, but there would be little explanation as to the purpose of the activities or the 
rationale for the order of the sequence. Coaches noted that it would be beneficial if 
teachers took more time to articulate these connections for students, so that the students 
understood the purpose of the ERWC and the literacy strategies that it was intended to 
foster. For example, one coach noted, “[The teacher] and I talked about helping students 
to see the relevance of the work he was having them do. We talked about helping students 
to see the purpose of writing rhetorically ... that goes beyond the idea that they need to do 
this for school or for this class.” 

Another coach echoed this concern and partly attributed the “disconnection” to the 
approach that some teachers used for planning the course. This coach observed teachers 
in their PLC meetings and noticed that they planned the course one week at a time. The 
coach stated that teachers seemed more concerned with pacing the course than with the 
overall intentions of the ERWC curriculum, resulting in what the coach called a “non- 
integrated” presentation of the modules: 

The teacher can’t let go of managing the class, delivering information, maintaining 
order, or being the source of information in the class. [The teacher is] still functioning 
from a coverage model for instruction. This is [a] very teacher-centered class. Students 
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are being directed through the “packets” and the teacher feels an obligation to meet the 
pacing ideas developed in the ERWC PLC that meets each week to determine what is 
going to be covered the next week. It is a week-to-week planning session, not a module- 
planning session. That focus reinforces the non-integrated presentation of the modules 
that I am seeing on this campus — and [on] others. 

Coaches also documented instances in which teachers modified the ERWC curriculum in 
ways that did not seem aligned to the ERWC philosophy. In one example offered by a 
coach, the teacher modified an activity in a module because she did not have enough time. 
Instead of having students participate in a panel discussion with the entire class, as 
suggested in the module, the teacher had the students work in small groups. The coach’s 
concern, documented over a series of coaching logs for this particular teacher, was that 
this emphasis on small-group activities was resulting in less student engagement and 
learning. Identifying what he saw as a pacing issue, the coach suggested that the teacher 
spend less time on small-group activities, in order to accommodate the panel discussion, 
with the goal of increasing student voice and engagement: 

I discussed the “time crunch ” (the teacher’s term) with her after class and she explained 
how she had modified the ending activity because the panel would have taken too much 
time. I encouraged her in her subsequent class to take less time with the initial group 
activity so that she could have more time for the panel, which would stimulate more 
student speaking and have increased potential for engagement and learning. 

Some teachers’ low expectations of students, reflected in low accountability regarding the 
work and behavior required of students, were another pedagogical challenge that some 
coaches noticed. Evidence of this is provided by coaching logs that describe classroom 
scenarios or verbal interactions between coaches and teachers. For example: 

The issue of lowered expectations is also something I have written about regularly in this 
class. The teacher seems to have the impression that she will have more buy-in if she 
asks for minimal work. I also heard frustration in her voice when she said, early on, “Yes, 
guys, this is your personal invitation to open your notebook. Some of you seem to need 
that. ”... I haven’t had success in addressing [these] lowered expectations and this time 
didn’t bring it up. It seems like a belief that is stable and, therefore, unlikely to change 
and one that is persistent at this site across classes. Also, it’s a complex issue, because 
the skill, experience, and engagement level in this class is widely varied. Still, I feel like I 
should have done more to address it. 

Another example of low expectations was described by a coach who had observed a 
teacher asking students to write three sentences as a Quickwrite after watching the film 
version of “The Lottery.” The prompt provided by the teacher was “Tell me what you think 
is going on here in the story.” The coach expressed surprise at the teacher’s expectation for 
students to write only three sentences, as well as at the simplicity of the prompt, 
describing this activity as a “very low bar” considering that the students are high school 
seniors. 
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Finding 15: Coaches noted that the ERWC classes enrolled students with a wide 
range of ability levels and that this may have contributed to pacing challenges. 

Coaches noted that large differences in ability levels among students within the same 
classroom can lead to difficulties in pacing: 

Different levels in the class — so [some] students take a long time to do activities; others 
take [a] short time and are finished early. 

Reading the text to students aloud is not a recommended practice, but [the teacher] is 
concerned that students will not read otherwise. This goes to student placements in the 
course. 

These quotes suggest that students’ varying abilities pose a challenge for ERWC teachers. 
One coach perceived that some schools appear to be using the ERWC as a default 
remedial English course in the absence of remedial classes offered at the high school: 

Need to bring up the issue of uneven placements into ERWC at this school and 
throughout [the district]; in the absence of remedial courses, some schools that have 
ERWC seem to be using it as a default for all students. ... I have heard of this happening 
at [school name] as well as at [school name]. 

These challenges affect classroom management and pacing, requiring teachers to modify 
teaching strategies to meet the needs of all students. It is challenging for teachers to 
maintain the expected pace of the curriculum and implement activities with fidelity when 
there is a wide variety of student needs and abilities in the class. 

Professional Learning Communities 

As part of this project, each ERWC teacher participated in a professional learning 
community (PLC), an active community of learners. In their respective PLCs, teachers 
worked together to develop pacing for the curriculum; reflect on current practices; 
expand, refine, and build new skills; share ideas and teach one another; calibrate the 
scoring of student writing; monitor student learning; and collaboratively problem-solve 
challenges that arose. The PLCs were structured in different ways at different sites, 
depending on their different circumstances (e.g., numbers of teachers involved). Among 
the different configurations were a site-based ERWC teachers’ meeting; “critical friends,” 
two ERWC teachers who agreed to meet and support one other; a PLC that included 
ERWC teachers from multiple schools; and an ERWC PLC that, in addition to including 
ERWC teachers who were participating in this project, included ERWC teachers from the 
same school who were not participating in this project . 45 The ERWC coaches visited each 


45 ERWC teachers who were not participating in the evaluation were primarily those teaching the course 
for the first time. 
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PLC to which they were assigned at least twice during the year to offer assistance as 
needed. This section provides an overview of the main themes, successes, and challenges 
that emerged from the PLC meetings, as captured in the PLC logs and in end-of-year 
surveys. The template for the PLC log is provided in appendix B. 

Finding 16: Teachers reported valuing the opportunity to collaborate with 
colleagues through the ERWC PLCs. 

One teacher reported, “Our PLCs were very helpful — everyone participated, shared ideas 
and issues, problem solved, planned, etc. It created a solid community.” Another teacher 
reported the PLC time being used to “share trials, tribulations, and successes.” This 
collaboration and support positively affected teachers, leading one to write, “I felt inspired 
and as though I had guidance in how to implement the modules.” While the study 
requirement was that PLCs meet at least nine times over the course of the school year, 
some teachers mentioned that even more PLC time was needed. One wrote, “We needed 
more PLC time, especially when teaching 100% ERWC for the first time.” 

Finding 17: Teachers commonly used the PLC time to plan for upcoming modules. 

Early in each semester, PLC meetings commonly focused on mapping out the semester, 
deciding which modules would be taught, which writing assignments would be assigned, 
and approximately how long each module would take. According to the first log entry for 
one PLC, 

We decided on a shared curriculum map and chose eight modules to teach. ... We had a 
difficult time selecting our eight modules for various reasons. Some of us like different 
modules and many of our students have already been exposed to several modules in prior 
grades, so we had to decide which ones we would re-teach. We ended up being very 
happy with our course outline in the end. 

As noted earlier in this chapter, a commonly identified challenge for teachers throughout 
the year was time — not having enough time to fully implement a given module. Teachers 
noted that they often felt rushed to complete modules. For that reason, several PLCs 
worked on creating pacing guides for specific modules. Some of these efforts yielded 
“curriculum maps” or calendars that allotted specific time frames, usually set out in weeks, 
to complete a given module. Within a module’s time frame, teachers went further, 
identifying which activities should be completed by when and identifying which activities 
were optional. In one PLC, for example, teachers spent time going over module 1: 

We examined the articles and devised strategies that involved incorporating the articles, 
annotating activities, and exercises that worked toward the creation of a culminating 
college or career personal statement. 
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The following quote from one PLC log captures reflections on a recently completed 
module and the group’s high-level planning efforts for two upcoming modules: 

We met and discussed the pacing of the second module, Rhetoric of the Op-Ed. We 
agreed that next year we would try to take five rather than six weeks to complete this 
unit. We decided to assign the persuasive essay (Rhetoric of the Op-Ed) rather than the 
letter to the editor, as it was more academic. We also sketched out a plan to complete 
the Racial Profiling module in four weeks, asking students to research their topics over 
the Thanksgiving break and checking in with them when we return. This will allow us to 
do the Into the Wild module the week before we begin winter break, allowing students to 
work independently over the break, and then reviewing [student’s work] when we return. 

For modules that included a full-length book, teachers often worked together in their 
PLCs to create a timeline for students to finish reading specific sections of the book, 
breaking the reading into more manageable pieces for students. Another PLC worked on 
“chunking” sections of the book Into the Wild for module 6 to make it easier for students 
to get through, an effort that they found led to success not only in students’ 
understanding of the text, but in their ability to discuss and write about it: 

The ‘chunking’ we did for the reading was very successful and it made the book so much 
easier for the students to understand. It led to much better discussions amongst the 
students. We think the better discussions also led to students being more prepared to 
write their essays and students seemed to be able to discuss their thoughts better in their 
papers. 

As the school year began to wind down, teachers in PLC meetings strategized ways to keep 
students engaged in the course. Teachers in one PLC noted that “senioritis is kicking in 
and students are not keeping up with the readings.” Another noted, “We had to give daily 
reading quizzes ... to ensure students were keeping up with the reading.” 


Finding 18: Teachers collaborated in their PLCs to tailor the curriculum to meet 
their students’ needs. 

Responding to the common challenge of student deficits in specific skills needed for the 
course, teachers commonly spent time in the PLC meetings devising strategies to best 
scaffold the activities and assignments to support student understanding. One PLC log 
noted that, in April, “Writing topics in module needed to be scaffolded as assessed in 
student writing samples.” The teachers in another PLC documented how they spent the 
time together discussing and working out strategies to foster student engagement with the 
text, including connecting it to current issues and promoting class discussion. 

Teachers also noted students’ struggles with identifying solid counterarguments in 
module 2 , The Rhetoric of the Op-Ed Page, as well as their challenges in using a variety of 
sources in varied ways in their research papers. To address students’ difficulty with 
thinking of the persuasive agendas that writers of op-ed pieces use, and to identify flaws in 
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students’ logic, teachers in one PLC discussed how to “emphasize the metacognitive 
processes of reading opinion pieces to bring about a more thorough understanding and 
more powerful response” from students. 


Finding 19: Some PLCs spent time brainstorming how best to get students to 
understand the writing prompts and assignments. 

In a few PLCs, teachers noted that students were confused by some writing prompts or by 
ERWC directions. For those reasons, teachers in these PLCs spent time brainstorming how 
best to get students to understand the writing prompts and assignments. For example, one 
PLC examined a writing prompt in relation to students’ low grades on a writing exercise: 

We seem to have to review the directions with students in order to get the best response. 
The timed-writing is a prime example, which saw a majority of the students earn a C or 
D. After the exam, we reviewed the prompt with students, which revealed that many 
misread the prompt or did not know how to respond properly. 

Some teachers reported that they sometimes worked collaboratively with other PLC team 
members to decipher writing prompts themselves. Referring to prompts in module 4, one 
teacher wrote, “Sometimes, my team and I have to sit and puzzle over exactly what the 
prompts are asking the students to do.” 

Another teacher described that she and one of her fellow PLC members created tools to 
help students understand the prompt in module 1: 

The writing prompt was too vague for student comprehension. [We] reviewed with 
students a think aloud brainstorming chart created with a member of our PLC on how 
we thought through the writing prompt and connected it to the exercises completed in 
the module. [We] created sentence frames to assist students in understanding the task 
as identified in the prompt. Writing assignment could have suggested students refer to 
the admission requirements for the schools selected to ascertain whether or not personal 
essays were required for admission, and if so, what were the actual prompts. 


Finding 20: Teachers planned how to effectively assess students’ skills. 

In particular, teachers in the PLCs commonly worked on norming diagnostic essays. One 
PLC log described teachers’ efforts to norm student writing and establish an efficient 
grading process: 

We are norming our grading of essays amongst the five high schools in our district. We 
are still in the process of identifying student anchor papers for use during our district 
collaborative meetings. 


In some PLCs, teachers talked about how to streamline their grading process for the 
ERWC curriculum so that they could provide feedback to students in a timely manner. 
Grading student work in a timely fashion was especially challenging when teachers had 
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large class sizes. As teachers in one PLC wrote, “We are looking to maintain an efficient 
grading schedule so that formal writing assessments are graded and returned in a timely 
manner.” This challenge was noted across PLCs. 

During the PLC meetings, teachers also talked about strategies for addressing the issue of 
students not completing their readings on time. This led to creating short comprehension 
quizzes or having students write brief summaries and discussing them in class. 

Finding 21: One of the big challenges to PLC collaboration arose when teachers had 
different views of teaching in general and of the ERWC curriculum specifically. 

This challenge was documented by teachers in their end-of-year survey responses. As one 
teacher reflected, “One of the biggest challenges is dealing with inconsistencies in 
pedagogy. A couple of the teachers in my department do not share some of my 
fundamental views that are crucial to the ERWC theoretical foundations.” These 
inconsistencies related to the extent to which teachers believe in and apply key principles 
of the ERWC curriculum, including the integration of interactive reading and writing 
processes and a rhetorical approach that fosters critical thinking and engagement through 
a relentless focus on the text. 

Key to the ERWC model is having a student-centered approach to teaching, which 
responds to varied students’ needs and instructional contexts. Some teachers were not 
willing or able to shift away from a teacher-centered approach, in which students passively 
receive information, to a student-centered approach, in which students are active 
participants in their own learning. One teacher wrote, “I think that the buy-in and 
commitment to the program of the ERWC teachers is instrumental to the success of the 
students. It would be great if only those teachers excited about the program taught it.” 


67 


Chapter 6. Conclusions and 
Discussion 


This report provides results from an evaluation of the Expository Reading and Writing 
Course (ERWC)’s impact on student achievement at the end of grade 12. Twenty-four high 
schools throughout California participated in the evaluation during the 2013/14 school 
year. Students in the ERWC were compared with students who took a different grade-12 
English course — usually AP English Literature or English 4. Matching methods were 
employed to compare ERWC students with similar students who enrolled in a non-ERWC 
English course. The key outcome measure used to assess the effectiveness of the ERWC 
was the English Placement Test (EPT), which is the standardized assessment that the 
California State University uses to determine student eligibility for credit-bearing English 
courses. 

Results of the impact analysis found that ERWC students scored higher on the EPT than 
non-ERWC students, and that this result was statistically significant at the 1 percent level. 
The estimated effect size of the impact was 0.13 standard deviations. In addition, a number 
of sensitivity analyses were carried out to determine the robustness of the main impact 
findings. These sensitivity analyses included altering the methodology of the matching 
and altering the sample included in the analysis. Each sensitivity analysis produced similar 
results — positive and statistically significant — to the main findings. 

An assessment of implementation fidelity was also carried out, based on survey data 
obtained from study participants. While teacher participation was high in the 
professional-learning components (summer sessions, coaching, and professional learning 
communities), data from the implementation feedback charts revealed that teachers were 
not completing activities in all of the strands of the modules (Prereading, Reading, 
Postreading, Discovering What You Think, Entering the Conversation, and Revising and 
Editing). This is likely due to the fact that teachers were not instructed to teach at least 
one activity in each of the strands. Had this instruction been communicated to teachers at 
the beginning of the study, it seems likely that many more teachers would have met this 
requirement when they taught a module. In other words, if a teacher were to teach a given 
number of activities in a module, the teacher could have made sure that he or she taught 
at least one activity in each strand. 

An interesting topic for future exploration would be how implementation fidelity relates 
to academic outcomes — more specifically, whether the students of teachers who taught 
eight modules with fidelity over the course of the year demonstrated greater academic 
achievement. While an exploratory question such as this would only result in correlational 
conclusions rather than causal conclusions, it would nevertheless be worth exploring. 
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A related question pertains to what types of impact results would have been observed had 
more teachers taught eight modules with fidelity. 
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Appendix A. Statistical Power for 
Impact Estimates 


To determine the appropriate sample size required for the study, minimum detectible 
effect sizes (MDES) were calculated, which, as described in Schochet (2008), can be 
expressed as: 

MDES = Factor(a,fi,df)* y/Var (impact) /a 

where Var (impact) is the variance of the impact estimate, a is the standard deviation of 
the outcome measure, and F actor (.) is a constant that is a function of the significance level 
(a), the statistical power (P), and the number of degrees of freedom (df). For the power 
calculation, the following assumptions were made: a 5 percent significance level, power of 
0.8, a two-tailed test, and a sample size of 6,618 students who are randomly assigned to 
treatment or control groups. 46 

Based on these assumptions, the estimated MDES was .07. Overall, adequate statistical 
power was available for detecting small-sized impact estimates on student outcomes, due 
to the large number of students in the 24 study high schools. 


46 While the analytic design is a matching analysis where students self-selected into treatment and 
control groups (as opposed to being randomly assigned), we assumed that the power calculations were 
similar to that of a single-level student-randomized controlled trial. Our power analysis would produce 
MDESs that are slightly downward biased, but, given the large study sample and the low estimated 
MDES, this was not expected to materially impact the power calculation. 
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Appendix B. Data Collection Instruments to 
Measure Fidelity of Implementation 

Implementation Feedback Chart Instrument 


Teacher's First and Last Name: 

School Name : 

Date You Started Teaching This Module: 
Date You Finished Teaching This Module: 


Module 1: What's Next? Thinking About Life After High School 

Implementation Feedback Chart 


Frame: 

The purpose of this chart is to provide feedback on the module. This feedback will be used to understand how the module was taught 
and to further improve the module. 


Instructions: 

Please write your name, school name, start date, and end date at the top of the first page. 

Please fill out this chart as you progress through the module. You should submit it once you have finished the module. 
If you have completed the stated Activity, please check the "Completed" box. 
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If you have completed the Formative Assessment, please check the "Completed" box and indicate what you learned as a result in the 
"Notes" column. 

If you have completed the Rhetorical Grammar Activities, please check the "Completed" box and identify the Rhetorical Grammar 
Activity numbers that you completed in the "Notes" column. 


If you have not completed the stated Activity/Formative Assessment/Rhetorical Grammar Activity, please indicate the reason for not 
completing it in the "Not Completed" column using the following key: 

A = Students had already mastered this knowledge/skill 

B = I substituted this activity with another activity that was better suited for my students' needs 
C = I combined this activity with another activity in this module 
D = It was noted as an optional activity 
E = Our class did not have enough time to work on this activity 
F = Other (please note what it is in the "Notes" column) 


If you modified the Activity/Formative Assessment/Rhetorical Grammar Activity, please check the "Modified" box and indicate what 
change(s) you made in the "Notes" column. 

Please enter an estimate of the time spent using the scale 1-6, where 1=10 minutes, 2=20 minutes, 3=30 minutes, 4=40 minutes, 
5=50 minutes, and 6=60 minutes. 

Please enter an estimate of the level of depth achieved for each activity using the scale 1-3, where: 

l=Cursory (completed less than half of the items for the activity) 

2=Moderate (completed about half of the items for the activity) 

3=ln-depth (completed more than half of the items for the activity) 


In addition to describing any modifications or other reasons for not completing an activity, feel free to note any successes or challenges 
related to a specific activity in the "Notes" column. If the implementation varied across sections of ERWC you teach, please note in which ways it 
varied. 


After filling out the chart, please complete the open-ended Module Reflection that follows the chart. 


B-2 


Instructions for submitting this chart 


If you completed the hard copy version of this chart, please turn a copy of it in to your school's ERWC study site contact person and 
retain a copy for yourself. The study site contact person will collect the charts and mail them to WestEd. 

In order to complete this chart online using Survey Mon key, you will receive an email from SurveyMonkey (that is on behalf of Marie 
Olson at WestEd) that contains a link to the survey. Once you complete the chart in SurveyMonkey, it will automatically be sent to the WestEd 
evaluation team when you click the "Done" button on the last page of the survey. If you have any questions about completing the survey on-line 
or would like to review your responses to the chart after it has been submitted in SurveyMonkey, you may contact Marie Olson at 415-615-3371 
or molson2(5)wested.org . 

Please submit the completed chart within two weeks of finishing this module. 

If you have any questions about how to complete the information in this chart, please contact Laura Jaeger at 415-615-3382 or 
ljaeger@wested.org. 
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Activity 

Completed 

Not 

Completed 
(A, B, C, D, 
E, F) 

Modified 

Time 

(1-6) 

Level of 
Depth 
(1-3) 

Notes 

(successes, challenges or 
modifications; variation 
across sections) 

Example: 

Reading for Understanding 

s 



2 

3 

Students were very 
engaged with this 
particular reading; some 
vocabulary was challenging 
to the students. 

Reading Rhetorically 

Prereading 

Activity 1: Getting Ready to 
Read — An Overview of 
"What's Next? Thinking 
about Life After High 
School" 







Rhetorical Grammar 
Activities 







Activity 2: Activating Prior 
Knowledge 







Activity 3: Exploring Key 
Concepts 







Activity 4: Making 
Predictions and Asking 
Questions 







Activity 5: Understanding 
Key Vocabulary 







Text: "Want to Get into College? Learn to Fail" 
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Activity 

Completed 

Not 

Completed 
(A, B, C, D, 
E, F) 

Modified 

Time 

(1-6) 

Level of 
Depth 
(1-3) 

Notes 

(successes, challenges or 
modifications; variation 
across sections) 

Prereading 

Activity 6: Surveying the 
Text 







Reading 

Activity 7: Reading with the 
Grain 







Postreading 

Activity 8: Responding to 
Perez 







Text: "Hidden intellectualism" 

Prereading 

Activity 9: Getting Ready to 
Read and Exploring Key 
Concepts 







Activity 10: Understanding 
Key Vocabulary 







Formative Assessment 







Rhetorical Grammar 
Activities 







Reading 

Activity 11: Reading for 
Understanding — Stop and 
Respond 







Formative Assessment 
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Activity 

Completed 

Not 

Completed 
(A, B, C, D, 
E, F) 

Modified 

Time 

(1- 

6) 

Level of 
Depth 
(1-3) 

Notes 

(successes, challenges or 
modifications; variation 
across sections) 

Postreading 

Activity 12: Thinking 
Critically 







Activity 13: Summarizing 
and Responding 







Text: Jigsaw reading of four articles 



Prereading 

Activity 14: Surveying the 
Text 







Reading 

Activity 15: Reading for 
Understanding 







Postreading 

Activity 16: Summarizing 
and Responding 







Texts: "FAQ Guide for college or work" and "Web Site Resources" 

Prereading 

Activity 17: Making 
Predictions and Asking 
Questions 







Reading 
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Activity 

Completed 

Not 

Completed 
(A, B, C, D, 
E, F) 

Modified 

Time 

(1- 

6) 

Level of 
Depth 
(1-3) 

Notes 

(successes, challenges or 
modifications; variation 
across sections) 

Activity 18: Considering the 
Structure of a Web Site 







Activity 19: Reading for 
Understanding and 
Collecting Information 







Postreading 

Activity 20: Summarizing 
Research Findings 







Activity 21: Reflecting on 
Your Research Findings — 
Reading One Another's 
Findings 







Connecting Reading to Writing 

Discovering What You Think 

Activity 22: Considering the 
Writing Task 







Formative Assessment 







Activity 23: Taking a Stance 
— Elements of the 
Rhetorical Framework 







Formative Assessment 
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Activity 

Completed 

Not 

Completed 
(A, B, C, D, 
E,F) 

Modified 

Time 

(1- 

6) 

Level of 
Depth 
(1-3) 

Notes 

(successes, challenges or 
modifications; variation 
across sections) 

Writing Rhetorically 

Entering the Conversation 

Activity 24: Composing a 
Draft 







Activity 25: Considering 
Structure — Read Around 
Activity 







Formative Assessment 







Rhetorical Grammar 
Activities 







Revising and Editing 

Activity 26: Revising 
Rhetorically 







Activity 27: Editing 







Activity 28: Reflecting 
on Your Writing Process 
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What did you learn from teaching this module? 


Module Reflection 


What changes in your teaching would you make the next time you teach this module? 


How well do you think your students met the objectives of this module? What evidence did you draw on to assess 
whether students met the objectives? 


Were there any articles that you did not teach? Why? 


Is there anything else you would like to share regarding your experience teaching this module? 
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Coaching Log Instrument 


School: 

Principal: 

Contact Person: 

Teacher Contacted/Visited: 

Date: 

Time: 

Method of Contact: 

Email 

Skype 

Phone 

Visit 


Meeting Purpose: 


Classroom 

Meeting with 

PLC meeting at the 

Meeting with the 

PLC meeting 

observation 

individual teachers 

school site level 

administrator/ 

counselor 

(multi-school) 


Notes: 


Questions to answer after observation and post-conference: 
What is going well? 


What are some challenges that arose in the lesson? 


What steps are being taken to address these challenges? 


During the observed session, what was your sense of the students' engagement? To what do you 
attribute this level of engagement? 


What did the lesson do to help students become better readers and writers? 


Next steps: 


Next meeting date: 
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Professional Learning Community Log Instrument 

Expository Reading and Writing Course (ERWC) 

Professional Learning Community 
Attendance Log 

Instructions: Please have all PLC participants print and sign their names, and then please answer 
the following brief questions. Completed attendance logs should be turned in to the school site 
coordinator at the end of the month, who will then forward the documents to WestEd. It is 
expected that the ERWC PLC meetings should happen a minimum of once a month. 

School Name: Date of Meeting: 

Meeting Start Time: Meeting End Time: 


Teacher Name 

Signature 
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Questions to answer after the PLC meeting 


What topics were covered during the ERWC PLC meeting? 


What are some challenges that teachers are encountering? 


What successes are teachers experiencing with the ERWC curriculum? 


Next steps: 
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Appendix C. Rubric for Calculating Fidelity of 
Implementation for Each Component of the 
Expository Reading and Writing Course 


Component 
Number and 
Name 

Key 

Elements of 
Component 

Operational 
Definition for 
Indicator 

Data Source(s) for 
Measuring Indicator 

Scoring Criteria 

Scoring Criteria 
for Fidelity 

Program-Level 
Fidelity Threshold 

Component 1: 

Curriculum 

Materials 

ERWC 

Curriculum 

Materials 

Teacher receives all 
curriculum 
materials for the 
course. The 
curriculum 
materials are the 
teacher version of 
the curriculum, the 
student version of 
the curriculum, the 
student reader, 
and two full-length 
books. 

The teacher version of 
the curriculum will be 
received by teachers at 
the summer 
professional-learning 
sessions. The student 
version of the 
curriculum, the student 
reader, and the two full- 
length books will be sent 
to the schools, with 
communication with 
teachers to confirm 
receipt of materials. 

0=did not receive 
all materials; 

1 ^received all materials 
(Score range: 0 to 1) 

Teacher-level 
"With Adequate 
Fidelity” = 1 

Program-Level 
Score of “With 
Adequate Fidelity” 
= at least 95% of all 
ERWC study 
teachers receive a 
score of 1 . “Not 
With Adequate 
Fidelity” = less than 
95% of teachers 
receive a score 
of 1. 

Component 2: 

Teacher 

Professional 

Learning 

Sub-component: 

Summer 

Professional 

Learnina 

Two-day 
summer 
professional- 
learning session 

Number of summer 
professional- 
learning days that 
the teacher 
attends 

Attendance log from 
the implementation 
team 

0 (low)=attended 
zero days; 

1 (low)=attended one day 

2 (high)=attended two 
days 

(Score range: 0 to 2) 

Teacher-Level 
“With Adequate 
Fidelity” = 2 

Program-level 
Score of “With 
Adequate Fidelity” 
= at least 90% of all 
teachers receiving 
a score of 2 
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Component 
Number and 
Name 

Key 

Elements of 
Component 

Operational 
Definition for 
Indicator 

Data Source(s) for 
Measuring Indicator 

Scoring Criteria 

Scoring Criteria 
for Fidelity 

Program-Level 
Fidelity Threshold 

Component 2: 

Teacher 

Professional 

Learning 

Sub-component: 

Professional 

Learnina 

Community (PLC) 

Professional 

Learning 

Community 

Number of PLC 
meetings that the 
teacher attends 
over the school 
year 

Attendance logs from 
the PLC meetings 

0 (low)=attended zero to 
two PLC's; 

1 (low)=attended three or 
four PLC’s; 

2 (moderate)=attended 
five or six PLC's; 

3 (moderate)=attended 
seven or eight PLC's; 

4 (high)=attended nine or 
more PLC's 

(Score range: 0 to 4) 

Teacher-Level 
"With Adequate 
Fidelity” = 4 

Program-level 
Score of "With 
Adequate Fidelity" 
= at least 80% of all 
teachers receiving 
a score of 4 

Component 2: 
Teacher 
Professional 
Learning 

Sub-component: 

Coachina 

Sessions 

Coaching 

Sessions 

Number of 
coaching sessions 
that the teacher 
attends over the 
school year 

Attendance logs from 
coaches 

0 (low)=attended zero 
sessions; 

1 (low)=attended one 
session; 

2 (low)=attended two 
sessions; 

3 (moderate)=attended 
three sessions, 

4 (high)=attended four or 
more sessions 

(Score range: 0 to 4) 

Teacher-Level 
“With Adequate 
Fidelity” = 4 

Program-level 
Score of “With 
Adequate Fidelity" 
= at least 80% of all 
teachers receiving 
a score of 4 

Component 2: 
Teacher 
Professional 
Learning 

Two-day 
summer 
professional- 
learning session, 
PLC, and 
Coaching 
sessions 

Aggregate score 
comprised of 
summing the three 
sub-components 
for Component 2 

Data sources from the 
three sub-components 
for Component 2 

0 to 7 = low 
8 to 9 = moderate 
10= high 
(Score range: 

Oto 10) 

Teacher-level 
“With Adequate 
Fidelity" for this 
entire 

component is 
when the 
teacher scores at 
least a 9. 

Program-Level 
Score of “With 
Adequate Fidelity” 
for this entire 
component is if at 
least 80% of all 
ERWC teachers 
receive a score of 
at least 9. Program- 
Level Score of “Not 
With Adequate 
Fidelity" is if less 
than 80% of all 
teachers receive a 
score of at least 9. 
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Component 
Number and 
Name 

Key 

Elements of 
Component 

Operational 
Definition for 
Indicator 

Data Source(s) for 
Measuring Indicator 

Scoring Criteria 

Scoring Criteria 
for Fidelity 

Program-Level 
Fidelity Threshold 

Component 3: 
Implementation of 
ERWC Curriculum 

Implementation 
of ERWC 
Curriculum 

Number of ERWC 
modules that the 
teacher taught 
with adequate 
fidelity during the 
school year (out of 
1 2 possible 
modules) 

Teacher survey that 
teachers fill out and 
return within two weeks 
of completing each 
module (there is one 
survey for each 
module) 

Score = number of modules 
completed with adequate 
fidelity 

(Score range: 0 to 12) 

Teacher-Level 
With Adequate 
Fidelity = 8 

Program-Level 
Score of “With 
Adequate Fidelity” 
= at least 80% of all 
teachers score at 
least 8. Program- 
Level Score of “Not 
With Adequate 
Fidelity” = less than 
80% of all teachers 
score at least 8. 


Source: Authors. 
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